2024-04-25T10:52:48.898 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T10:52:48.898 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/cap-flush} --archive /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673109 --name pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi --verbose --owner scheduled_pdonnell@teuthology -- /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673109/orig.config.yaml 2024-04-25T10:52:48.950 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T10:52:49.045 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673109 branch: wip-pdonnell-testing-20240425.015853-debug description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/cap-flush} email: pdonnell@redhat.com first_in_suite: false job_id: '7673109' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240425.015853-debug ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Replacing daemon mds.a sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 kclient: syntax: v2 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240425.015853-debug sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 owner: scheduled_pdonnell@teuthology priority: 85 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1880 sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 sleep_before_teardown: 0 subset: 178/512 suite: fs suite_branch: wip-pdonnell-testing-20240425.015853-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi069.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPVaRnKTw/F4gPOhcn3MJrgOv1TKo70DqBUsU44sG8l0vXWamAe8zF1gjRru8pRNCoutSKLlssRFrinEGkP+0u0= smithi146.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDFfpZqoB7XKebiOPIa8TY4It0GGcnaZ4m6vSuJ68mo8RrxCBAwGSzawPnCWbGI1cP+kSU7K9iuysYNuXHZK008= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_cap_flush teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_10:00:23 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T10:52:49.046 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa; will attempt to use it 2024-04-25T10:52:49.047 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks 2024-04-25T10:52:49.047 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T10:52:49.049 INFO:teuthology.task.internal:Checking packages... 2024-04-25T10:52:49.081 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '994e2f20102a20fa9a27966c14b625cc01884df8' 2024-04-25T10:52:49.081 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T10:52:49.081 INFO:teuthology.packaging:ref: None 2024-04-25T10:52:49.082 INFO:teuthology.packaging:tag: None 2024-04-25T10:52:49.082 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240425.015853-debug 2024-04-25T10:52:49.082 INFO:teuthology.packaging:sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:52:49.082 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240425.015853-debug 2024-04-25T10:52:49.220 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3291.g994e2f20 2024-04-25T10:52:49.222 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T10:52:49.229 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T10:52:49.229 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T10:52:49.264 INFO:teuthology.task.internal:Saving configuration 2024-04-25T10:52:49.281 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T10:52:49.319 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T10:52:49.345 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi069.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673109', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 10:47:25.632288', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPVaRnKTw/F4gPOhcn3MJrgOv1TKo70DqBUsU44sG8l0vXWamAe8zF1gjRru8pRNCoutSKLlssRFrinEGkP+0u0='} 2024-04-25T10:52:49.366 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi146.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673109', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 10:47:25.633854', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDFfpZqoB7XKebiOPIa8TY4It0GGcnaZ4m6vSuJ68mo8RrxCBAwGSzawPnCWbGI1cP+kSU7K9iuysYNuXHZK008='} 2024-04-25T10:52:49.366 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T10:52:49.372 INFO:teuthology.task.internal:roles: ubuntu@smithi069.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-25T10:52:49.372 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-25T10:52:49.373 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T10:52:49.468 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f46fc7cbf70>, signals=[15]) 2024-04-25T10:52:49.468 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T10:52:49.476 INFO:teuthology.task.internal:Opening connections... 2024-04-25T10:52:49.476 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi069.front.sepia.ceph.com 2024-04-25T10:52:49.479 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:52:49.553 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi146.front.sepia.ceph.com 2024-04-25T10:52:49.554 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:52:49.629 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T10:52:49.636 DEBUG:teuthology.orchestra.run.smithi069:> uname -m 2024-04-25T10:52:49.654 INFO:teuthology.orchestra.run.smithi069.stdout:x86_64 2024-04-25T10:52:49.654 DEBUG:teuthology.orchestra.run.smithi069:> cat /etc/os-release 2024-04-25T10:52:49.711 INFO:teuthology.orchestra.run.smithi069.stdout:NAME="CentOS Stream" 2024-04-25T10:52:49.711 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION="9" 2024-04-25T10:52:49.711 INFO:teuthology.orchestra.run.smithi069.stdout:ID="centos" 2024-04-25T10:52:49.711 INFO:teuthology.orchestra.run.smithi069.stdout:ID_LIKE="rhel fedora" 2024-04-25T10:52:49.711 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION_ID="9" 2024-04-25T10:52:49.711 INFO:teuthology.orchestra.run.smithi069.stdout:PLATFORM_ID="platform:el9" 2024-04-25T10:52:49.711 INFO:teuthology.orchestra.run.smithi069.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T10:52:49.711 INFO:teuthology.orchestra.run.smithi069.stdout:ANSI_COLOR="0;31" 2024-04-25T10:52:49.712 INFO:teuthology.orchestra.run.smithi069.stdout:LOGO="fedora-logo-icon" 2024-04-25T10:52:49.712 INFO:teuthology.orchestra.run.smithi069.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T10:52:49.712 INFO:teuthology.orchestra.run.smithi069.stdout:HOME_URL="https://centos.org/" 2024-04-25T10:52:49.712 INFO:teuthology.orchestra.run.smithi069.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T10:52:49.712 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T10:52:49.712 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T10:52:49.712 INFO:teuthology.lock.ops:Updating smithi069.front.sepia.ceph.com on lock server 2024-04-25T10:52:49.731 DEBUG:teuthology.orchestra.run.smithi146:> uname -m 2024-04-25T10:52:49.749 INFO:teuthology.orchestra.run.smithi146.stdout:x86_64 2024-04-25T10:52:49.749 DEBUG:teuthology.orchestra.run.smithi146:> cat /etc/os-release 2024-04-25T10:52:49.807 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="CentOS Stream" 2024-04-25T10:52:49.807 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="9" 2024-04-25T10:52:49.808 INFO:teuthology.orchestra.run.smithi146.stdout:ID="centos" 2024-04-25T10:52:49.808 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE="rhel fedora" 2024-04-25T10:52:49.808 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="9" 2024-04-25T10:52:49.808 INFO:teuthology.orchestra.run.smithi146.stdout:PLATFORM_ID="platform:el9" 2024-04-25T10:52:49.808 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T10:52:49.808 INFO:teuthology.orchestra.run.smithi146.stdout:ANSI_COLOR="0;31" 2024-04-25T10:52:49.809 INFO:teuthology.orchestra.run.smithi146.stdout:LOGO="fedora-logo-icon" 2024-04-25T10:52:49.809 INFO:teuthology.orchestra.run.smithi146.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T10:52:49.809 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://centos.org/" 2024-04-25T10:52:49.809 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T10:52:49.809 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T10:52:49.809 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T10:52:49.810 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server 2024-04-25T10:52:49.831 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T10:52:49.840 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T10:52:49.873 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T10:52:49.874 DEBUG:teuthology.orchestra.run.smithi069:> test '!' -e /home/ubuntu/cephtest 2024-04-25T10:52:49.876 DEBUG:teuthology.orchestra.run.smithi146:> test '!' -e /home/ubuntu/cephtest 2024-04-25T10:52:49.892 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T10:52:49.906 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T10:52:49.906 DEBUG:teuthology.orchestra.run.smithi069:> test -z $(ls -A /var/lib/ceph) 2024-04-25T10:52:49.932 DEBUG:teuthology.orchestra.run.smithi146:> test -z $(ls -A /var/lib/ceph) 2024-04-25T10:52:49.995 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T10:52:50.045 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T10:52:50.060 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T10:52:50.060 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-04-25T10:52:50.060 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-04-25T10:52:50.061 DEBUG:teuthology.orchestra.run.smithi069:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T10:52:50.061 DEBUG:teuthology.orchestra.run.smithi146:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T10:52:50.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:52:50.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:52:50.078 DEBUG:teuthology.orchestra.run.smithi069:> uname -r 2024-04-25T10:52:50.079 DEBUG:teuthology.orchestra.run.smithi146:> uname -r 2024-04-25T10:52:50.135 INFO:teuthology.orchestra.run.smithi069.stdout:5.14.0-437.el9.x86_64 2024-04-25T10:52:50.135 INFO:teuthology.task.kernel:Running kernel on smithi069: 5.14.0-437.el9.x86_64 2024-04-25T10:52:50.136 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum install -y kernel 2024-04-25T10:52:50.137 INFO:teuthology.orchestra.run.smithi146.stdout:5.14.0-437.el9.x86_64 2024-04-25T10:52:50.137 INFO:teuthology.task.kernel:Running kernel on smithi146: 5.14.0-437.el9.x86_64 2024-04-25T10:52:50.137 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum install -y kernel 2024-04-25T10:52:52.315 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-25T10:52:52.479 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-25T10:52:55.013 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-04-25T10:52:55.782 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - AppStream 7.9 MB/s | 19 MB 00:02 2024-04-25T10:52:59.355 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - CRB 8.2 MB/s | 6.2 MB 00:00 2024-04-25T10:53:00.132 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - CRB 7.8 MB/s | 6.2 MB 00:00 2024-04-25T10:53:00.998 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 16 kB 00:00 2024-04-25T10:53:01.775 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 16 kB 00:00 2024-04-25T10:53:02.293 INFO:teuthology.orchestra.run.smithi069.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-04-25T10:53:02.876 INFO:teuthology.orchestra.run.smithi146.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:00 2024-04-25T10:53:07.357 INFO:teuthology.orchestra.run.smithi069.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-25T10:53:08.034 INFO:teuthology.orchestra.run.smithi146.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-04-25T10:53:09.246 INFO:teuthology.orchestra.run.smithi069.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T10:53:09.246 INFO:teuthology.orchestra.run.smithi069.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T10:53:09.332 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-04-25T10:53:09.333 INFO:teuthology.orchestra.run.smithi069.stdout:Nothing to do. 2024-04-25T10:53:09.333 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-04-25T10:53:09.430 DEBUG:teuthology.orchestra.run.smithi069:> echo no | sudo yum reinstall kernel || true 2024-04-25T10:53:09.932 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:00:02 ago on Thu 25 Apr 2024 10:53:07 AM UTC. 2024-04-25T10:53:09.942 INFO:teuthology.orchestra.run.smithi146.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T10:53:09.942 INFO:teuthology.orchestra.run.smithi146.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T10:53:10.038 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-04-25T10:53:10.039 INFO:teuthology.orchestra.run.smithi146.stdout:Nothing to do. 2024-04-25T10:53:10.039 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-04-25T10:53:10.061 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-04-25T10:53:10.070 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-04-25T10:53:10.070 INFO:teuthology.orchestra.run.smithi069.stdout: Package Architecture Version Repository Size 2024-04-25T10:53:10.070 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-04-25T10:53:10.070 INFO:teuthology.orchestra.run.smithi069.stdout:Reinstalling: 2024-04-25T10:53:10.070 INFO:teuthology.orchestra.run.smithi069.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T10:53:10.071 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:53:10.071 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2024-04-25T10:53:10.071 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-04-25T10:53:10.071 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:53:10.071 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 255 k 2024-04-25T10:53:10.071 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 0 2024-04-25T10:53:10.071 INFO:teuthology.orchestra.run.smithi069.stderr:Operation aborted. 2024-04-25T10:53:10.127 INFO:teuthology.orchestra.run.smithi069.stdout:Is this ok [y/N]: 2024-04-25T10:53:10.127 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum reinstall -y kernel || true 2024-04-25T10:53:10.150 DEBUG:teuthology.orchestra.run.smithi146:> echo no | sudo yum reinstall kernel || true 2024-04-25T10:53:10.614 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 10:53:07 AM UTC. 2024-04-25T10:53:10.648 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:00:02 ago on Thu 25 Apr 2024 10:53:08 AM UTC. 2024-04-25T10:53:10.730 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-04-25T10:53:10.731 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-04-25T10:53:10.731 INFO:teuthology.orchestra.run.smithi069.stdout: Package Architecture Version Repository Size 2024-04-25T10:53:10.731 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-04-25T10:53:10.731 INFO:teuthology.orchestra.run.smithi069.stdout:Reinstalling: 2024-04-25T10:53:10.731 INFO:teuthology.orchestra.run.smithi069.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T10:53:10.731 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:53:10.731 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2024-04-25T10:53:10.731 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-04-25T10:53:10.732 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:53:10.732 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 255 k 2024-04-25T10:53:10.732 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 0 2024-04-25T10:53:10.732 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2024-04-25T10:53:10.766 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-04-25T10:53:10.776 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-04-25T10:53:10.776 INFO:teuthology.orchestra.run.smithi146.stdout: Package Architecture Version Repository Size 2024-04-25T10:53:10.776 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-04-25T10:53:10.776 INFO:teuthology.orchestra.run.smithi146.stdout:Reinstalling: 2024-04-25T10:53:10.776 INFO:teuthology.orchestra.run.smithi146.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T10:53:10.776 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:53:10.776 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-04-25T10:53:10.776 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-04-25T10:53:10.777 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:53:10.777 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 255 k 2024-04-25T10:53:10.777 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 0 2024-04-25T10:53:10.777 INFO:teuthology.orchestra.run.smithi146.stderr:Operation aborted. 2024-04-25T10:53:10.827 INFO:teuthology.orchestra.run.smithi146.stdout:Is this ok [y/N]: 2024-04-25T10:53:10.827 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum reinstall -y kernel || true 2024-04-25T10:53:11.126 INFO:teuthology.orchestra.run.smithi069.stdout:kernel-5.14.0-437.el9.x86_64.rpm 901 kB/s | 255 kB 00:00 2024-04-25T10:53:11.126 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:53:11.126 INFO:teuthology.orchestra.run.smithi069.stdout:Total 646 kB/s | 255 kB 00:00 2024-04-25T10:53:11.146 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2024-04-25T10:53:11.319 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 10:53:08 AM UTC. 2024-04-25T10:53:11.322 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2024-04-25T10:53:11.322 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2024-04-25T10:53:11.327 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2024-04-25T10:53:11.327 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2024-04-25T10:53:11.436 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-04-25T10:53:11.437 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-04-25T10:53:11.437 INFO:teuthology.orchestra.run.smithi146.stdout: Package Architecture Version Repository Size 2024-04-25T10:53:11.437 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-04-25T10:53:11.437 INFO:teuthology.orchestra.run.smithi146.stdout:Reinstalling: 2024-04-25T10:53:11.437 INFO:teuthology.orchestra.run.smithi146.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T10:53:11.438 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:53:11.438 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-04-25T10:53:11.438 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-04-25T10:53:11.438 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:53:11.438 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 255 k 2024-04-25T10:53:11.438 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 0 2024-04-25T10:53:11.438 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-04-25T10:53:11.463 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2024-04-25T10:53:11.543 INFO:teuthology.orchestra.run.smithi069.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T10:53:11.544 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T10:53:12.150 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T10:53:12.150 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T10:53:12.180 INFO:teuthology.orchestra.run.smithi146.stdout:kernel-5.14.0-437.el9.x86_64.rpm 556 kB/s | 255 kB 00:00 2024-04-25T10:53:12.180 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:53:12.180 INFO:teuthology.orchestra.run.smithi146.stdout:Total 343 kB/s | 255 kB 00:00 2024-04-25T10:53:12.202 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-04-25T10:53:12.383 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-04-25T10:53:12.383 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-04-25T10:53:12.388 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-04-25T10:53:12.388 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-04-25T10:53:12.524 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-04-25T10:53:12.598 INFO:teuthology.orchestra.run.smithi146.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T10:53:12.598 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T10:53:12.826 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T10:53:12.826 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:53:12.826 INFO:teuthology.orchestra.run.smithi069.stdout:Reinstalled: 2024-04-25T10:53:12.826 INFO:teuthology.orchestra.run.smithi069.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T10:53:12.826 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:53:12.826 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-04-25T10:53:12.979 DEBUG:teuthology.orchestra.run.smithi069:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T10:53:13.025 INFO:teuthology.orchestra.run.smithi069.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T10:53:13.025 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T10:53:13.026 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T10:53:13.026 DEBUG:teuthology.parallel:result is None 2024-04-25T10:53:13.262 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T10:53:13.262 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T10:53:13.992 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T10:53:13.992 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:53:13.993 INFO:teuthology.orchestra.run.smithi146.stdout:Reinstalled: 2024-04-25T10:53:13.993 INFO:teuthology.orchestra.run.smithi146.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T10:53:13.993 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:53:13.993 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-04-25T10:53:14.166 DEBUG:teuthology.orchestra.run.smithi146:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T10:53:14.255 INFO:teuthology.orchestra.run.smithi146.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T10:53:14.255 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T10:53:14.255 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T10:53:14.255 DEBUG:teuthology.parallel:result is None 2024-04-25T10:53:14.255 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T10:53:14.263 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T10:53:14.263 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T10:53:14.265 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T10:53:14.281 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T10:53:14.295 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T10:53:14.346 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T10:53:14.346 DEBUG:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T10:53:14.348 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T10:53:14.375 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T10:53:14.396 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T10:53:14.396 DEBUG:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T10:53:14.413 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T10:53:14.446 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:53:14.447 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:53:14.458 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:53:14.466 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:53:14.468 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T10:53:14.475 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T10:53:14.475 DEBUG:teuthology.orchestra.run.smithi069:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T10:53:14.513 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T10:53:14.537 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T10:53:14.546 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T10:53:14.547 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T10:53:14.580 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T10:53:14.594 DEBUG:teuthology.orchestra.run.smithi069:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T10:53:14.676 DEBUG:teuthology.orchestra.run.smithi069:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T10:53:14.735 DEBUG:teuthology.orchestra.run.smithi069:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T10:53:14.816 DEBUG:teuthology.orchestra.run.smithi069:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T10:53:14.856 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:53:14.856 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T10:53:14.920 DEBUG:teuthology.orchestra.run.smithi146:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T10:53:14.960 DEBUG:teuthology.orchestra.run.smithi146:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T10:53:15.010 DEBUG:teuthology.orchestra.run.smithi146:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T10:53:15.092 DEBUG:teuthology.orchestra.run.smithi146:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T10:53:15.131 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T10:53:15.131 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T10:53:15.194 DEBUG:teuthology.orchestra.run.smithi069:> sudo service rsyslog restart 2024-04-25T10:53:15.197 DEBUG:teuthology.orchestra.run.smithi146:> sudo service rsyslog restart 2024-04-25T10:53:15.235 INFO:teuthology.orchestra.run.smithi069.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:53:15.266 INFO:teuthology.orchestra.run.smithi146.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:53:15.608 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T10:53:15.616 INFO:teuthology.task.internal:Starting timer... 2024-04-25T10:53:15.616 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T10:53:15.650 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T10:53:15.699 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-25T10:53:15.699 DEBUG:teuthology.orchestra.run.smithi069:> sudo service auditd rotate 2024-04-25T10:53:15.755 INFO:teuthology.orchestra.run.smithi069.stdout:Rotating logs: 2024-04-25T10:53:15.757 DEBUG:teuthology.orchestra.run.smithi146:> sudo service auditd rotate 2024-04-25T10:53:15.824 INFO:teuthology.orchestra.run.smithi146.stdout:Rotating logs: 2024-04-25T10:53:15.826 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T10:53:15.826 DEBUG:teuthology.orchestra.run.smithi069:> /usr/sbin/getenforce 2024-04-25T10:53:15.846 INFO:teuthology.orchestra.run.smithi069.stdout:Permissive 2024-04-25T10:53:15.847 DEBUG:teuthology.orchestra.run.smithi146:> /usr/sbin/getenforce 2024-04-25T10:53:15.867 INFO:teuthology.orchestra.run.smithi146.stdout:Permissive 2024-04-25T10:53:15.867 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi069.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi146.front.sepia.ceph.com': 'permissive'} 2024-04-25T10:53:15.867 DEBUG:teuthology.orchestra.run.smithi069:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T10:53:15.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:53:15.911 DEBUG:teuthology.orchestra.run.smithi146:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T10:53:15.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:53:15.935 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T10:53:15.935 DEBUG:teuthology.orchestra.run.smithi069:> sudo /usr/sbin/setenforce permissive 2024-04-25T10:53:15.975 DEBUG:teuthology.orchestra.run.smithi146:> sudo /usr/sbin/setenforce permissive 2024-04-25T10:53:16.000 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T10:53:16.009 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T10:53:16.014 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T10:53:16.138 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T10:53:16.156 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T10:53:16.157 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi069.front.sepia.ceph.com,smithi146.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T10:56:42.550 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi069.front.sepia.ceph.com'), Remote(name='ubuntu@smithi146.front.sepia.ceph.com')] 2024-04-25T10:56:42.551 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-04-25T10:56:42.553 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:56:42.626 DEBUG:teuthology.orchestra.run.smithi069:> true 2024-04-25T10:56:42.705 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-04-25T10:56:42.706 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-04-25T10:56:42.706 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:56:42.781 DEBUG:teuthology.orchestra.run.smithi146:> true 2024-04-25T10:56:42.817 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-04-25T10:56:42.818 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T10:56:42.828 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T10:56:42.829 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T10:56:42.829 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:56:42.831 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T10:56:42.831 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:56:42.856 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T10:56:42.869 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T10:56:42.885 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T10:56:42.891 INFO:teuthology.orchestra.run.smithi069.stderr:sudo: ntpd: command not found 2024-04-25T10:56:42.899 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T10:56:42.901 INFO:teuthology.orchestra.run.smithi069.stdout:506 Cannot talk to daemon 2024-04-25T10:56:42.914 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T10:56:42.921 INFO:teuthology.orchestra.run.smithi146.stderr:sudo: ntpd: command not found 2024-04-25T10:56:42.927 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T10:56:42.931 INFO:teuthology.orchestra.run.smithi146.stdout:506 Cannot talk to daemon 2024-04-25T10:56:42.944 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T10:56:42.957 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T10:56:42.981 INFO:teuthology.orchestra.run.smithi069.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:56:42.985 INFO:teuthology.orchestra.run.smithi069.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:56:42.985 INFO:teuthology.orchestra.run.smithi069.stdout:=============================================================================== 2024-04-25T10:56:42.985 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:56:42.985 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:56:42.985 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:56:42.985 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:56:43.011 INFO:teuthology.orchestra.run.smithi146.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:56:43.015 INFO:teuthology.orchestra.run.smithi146.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:56:43.015 INFO:teuthology.orchestra.run.smithi146.stdout:=============================================================================== 2024-04-25T10:56:43.015 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:56:43.015 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:56:43.015 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:56:43.015 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:56:43.016 INFO:teuthology.run_tasks:Running task install... 2024-04-25T10:56:43.024 DEBUG:teuthology.task.install:project ceph 2024-04-25T10:56:43.024 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}} 2024-04-25T10:56:43.025 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'} 2024-04-25T10:56:43.025 INFO:teuthology.task.install:Using flavor: default 2024-04-25T10:56:43.030 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T10:56:43.030 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T10:56:43.030 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T10:56:43.031 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:56:43.033 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T10:56:43.033 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:56:43.257 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T10:56:43.258 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T10:56:43.261 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T10:56:43.261 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T10:56:43.426 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T10:56:43.426 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T10:56:43.426 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T10:56:43.433 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T10:56:43.433 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:56:43.433 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T10:56:43.451 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T10:56:43.451 DEBUG:teuthology.orchestra.run.smithi146:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T10:56:43.459 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T10:56:43.459 DEBUG:teuthology.orchestra.run.smithi069:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T10:56:43.518 DEBUG:teuthology.orchestra.run.smithi146:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T10:56:43.525 DEBUG:teuthology.orchestra.run.smithi069:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T10:56:43.591 DEBUG:teuthology.orchestra.run.smithi146:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T10:56:43.600 DEBUG:teuthology.orchestra.run.smithi069:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T10:56:43.653 INFO:teuthology.orchestra.run.smithi146.stdout:check_obsoletes = 1 2024-04-25T10:56:43.655 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum clean all 2024-04-25T10:56:43.663 INFO:teuthology.orchestra.run.smithi069.stdout:check_obsoletes = 1 2024-04-25T10:56:43.665 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum clean all 2024-04-25T10:56:43.880 INFO:teuthology.orchestra.run.smithi146.stdout:48 files removed 2024-04-25T10:56:43.889 INFO:teuthology.orchestra.run.smithi069.stdout:48 files removed 2024-04-25T10:56:43.907 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T10:56:43.916 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T10:56:44.525 INFO:teuthology.orchestra.run.smithi069.stdout:ceph packages for x86_64 239 kB/s | 84 kB 00:00 2024-04-25T10:56:44.645 INFO:teuthology.orchestra.run.smithi146.stdout:ceph packages for x86_64 175 kB/s | 84 kB 00:00 2024-04-25T10:56:44.859 INFO:teuthology.orchestra.run.smithi069.stdout:ceph noarch packages 40 kB/s | 12 kB 00:00 2024-04-25T10:56:45.021 INFO:teuthology.orchestra.run.smithi146.stdout:ceph noarch packages 35 kB/s | 12 kB 00:00 2024-04-25T10:56:45.293 INFO:teuthology.orchestra.run.smithi069.stdout:ceph source packages 4.8 kB/s | 1.9 kB 00:00 2024-04-25T10:56:45.454 INFO:teuthology.orchestra.run.smithi146.stdout:ceph source packages 4.8 kB/s | 1.9 kB 00:00 2024-04-25T10:56:45.746 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.0 MB 00:00 2024-04-25T10:56:45.929 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.0 MB 00:00 2024-04-25T10:56:47.636 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-04-25T10:56:47.864 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-25T10:56:51.962 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - CRB 8.2 MB/s | 6.2 MB 00:00 2024-04-25T10:56:52.064 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-25T10:56:53.613 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-04-25T10:56:53.708 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 16 kB 00:00 2024-04-25T10:56:54.975 INFO:teuthology.orchestra.run.smithi069.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-04-25T10:56:54.992 INFO:teuthology.orchestra.run.smithi146.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-04-25T10:56:59.998 INFO:teuthology.orchestra.run.smithi069.stdout:lab-extras 40 kB/s | 1.7 kB 00:00 2024-04-25T10:57:00.033 INFO:teuthology.orchestra.run.smithi146.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-25T10:57:02.123 INFO:teuthology.orchestra.run.smithi069.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T10:57:02.124 INFO:teuthology.orchestra.run.smithi069.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T10:57:02.135 INFO:teuthology.orchestra.run.smithi069.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T10:57:02.135 INFO:teuthology.orchestra.run.smithi069.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T10:57:02.136 INFO:teuthology.orchestra.run.smithi069.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T10:57:02.137 INFO:teuthology.orchestra.run.smithi069.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T10:57:02.138 INFO:teuthology.orchestra.run.smithi069.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T10:57:02.141 INFO:teuthology.orchestra.run.smithi069.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T10:57:02.142 INFO:teuthology.orchestra.run.smithi069.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T10:57:02.185 INFO:teuthology.orchestra.run.smithi146.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T10:57:02.186 INFO:teuthology.orchestra.run.smithi146.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T10:57:02.196 INFO:teuthology.orchestra.run.smithi146.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T10:57:02.197 INFO:teuthology.orchestra.run.smithi146.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T10:57:02.198 INFO:teuthology.orchestra.run.smithi146.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T10:57:02.199 INFO:teuthology.orchestra.run.smithi146.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T10:57:02.200 INFO:teuthology.orchestra.run.smithi146.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T10:57:02.201 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-04-25T10:57:02.202 INFO:teuthology.orchestra.run.smithi146.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T10:57:02.203 INFO:teuthology.orchestra.run.smithi146.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T10:57:02.210 INFO:teuthology.orchestra.run.smithi069.stdout:======================================================================================= 2024-04-25T10:57:02.210 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository Size 2024-04-25T10:57:02.210 INFO:teuthology.orchestra.run.smithi069.stdout:======================================================================================= 2024-04-25T10:57:02.210 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2024-04-25T10:57:02.211 INFO:teuthology.orchestra.run.smithi069.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T10:57:02.211 INFO:teuthology.orchestra.run.smithi069.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T10:57:02.211 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T10:57:02.211 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T10:57:02.211 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T10:57:02.211 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T10:57:02.211 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T10:57:02.211 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T10:57:02.211 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T10:57:02.211 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T10:57:02.211 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T10:57:02.211 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T10:57:02.212 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T10:57:02.212 INFO:teuthology.orchestra.run.smithi069.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T10:57:02.212 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T10:57:02.212 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T10:57:02.212 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T10:57:02.212 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T10:57:02.212 INFO:teuthology.orchestra.run.smithi069.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T10:57:02.212 INFO:teuthology.orchestra.run.smithi069.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T10:57:02.212 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T10:57:02.212 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T10:57:02.212 INFO:teuthology.orchestra.run.smithi069.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T10:57:02.212 INFO:teuthology.orchestra.run.smithi069.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T10:57:02.212 INFO:teuthology.orchestra.run.smithi069.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T10:57:02.212 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T10:57:02.212 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T10:57:02.213 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T10:57:02.213 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T10:57:02.213 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T10:57:02.213 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T10:57:02.213 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T10:57:02.213 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T10:57:02.213 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T10:57:02.213 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T10:57:02.213 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T10:57:02.213 INFO:teuthology.orchestra.run.smithi069.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T10:57:02.213 INFO:teuthology.orchestra.run.smithi069.stdout:Upgrading: 2024-04-25T10:57:02.213 INFO:teuthology.orchestra.run.smithi069.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T10:57:02.214 INFO:teuthology.orchestra.run.smithi069.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T10:57:02.214 INFO:teuthology.orchestra.run.smithi069.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T10:57:02.214 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T10:57:02.214 INFO:teuthology.orchestra.run.smithi069.stdout:Installing dependencies: 2024-04-25T10:57:02.214 INFO:teuthology.orchestra.run.smithi069.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T10:57:02.214 INFO:teuthology.orchestra.run.smithi069.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T10:57:02.214 INFO:teuthology.orchestra.run.smithi069.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T10:57:02.214 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T10:57:02.214 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T10:57:02.214 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T10:57:02.214 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T10:57:02.214 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T10:57:02.215 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T10:57:02.215 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T10:57:02.215 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T10:57:02.215 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T10:57:02.215 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T10:57:02.215 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T10:57:02.215 INFO:teuthology.orchestra.run.smithi069.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T10:57:02.215 INFO:teuthology.orchestra.run.smithi069.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T10:57:02.215 INFO:teuthology.orchestra.run.smithi069.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T10:57:02.215 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T10:57:02.215 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T10:57:02.215 INFO:teuthology.orchestra.run.smithi069.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T10:57:02.215 INFO:teuthology.orchestra.run.smithi069.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T10:57:02.216 INFO:teuthology.orchestra.run.smithi069.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T10:57:02.216 INFO:teuthology.orchestra.run.smithi069.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T10:57:02.216 INFO:teuthology.orchestra.run.smithi069.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T10:57:02.216 INFO:teuthology.orchestra.run.smithi069.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T10:57:02.216 INFO:teuthology.orchestra.run.smithi069.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T10:57:02.216 INFO:teuthology.orchestra.run.smithi069.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T10:57:02.216 INFO:teuthology.orchestra.run.smithi069.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T10:57:02.216 INFO:teuthology.orchestra.run.smithi069.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T10:57:02.216 INFO:teuthology.orchestra.run.smithi069.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T10:57:02.216 INFO:teuthology.orchestra.run.smithi069.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T10:57:02.216 INFO:teuthology.orchestra.run.smithi069.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T10:57:02.216 INFO:teuthology.orchestra.run.smithi069.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T10:57:02.216 INFO:teuthology.orchestra.run.smithi069.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T10:57:02.217 INFO:teuthology.orchestra.run.smithi069.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T10:57:02.217 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T10:57:02.217 INFO:teuthology.orchestra.run.smithi069.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T10:57:02.217 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T10:57:02.217 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T10:57:02.217 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T10:57:02.217 INFO:teuthology.orchestra.run.smithi069.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T10:57:02.217 INFO:teuthology.orchestra.run.smithi069.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T10:57:02.217 INFO:teuthology.orchestra.run.smithi069.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T10:57:02.217 INFO:teuthology.orchestra.run.smithi069.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T10:57:02.217 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T10:57:02.217 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T10:57:02.217 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T10:57:02.218 INFO:teuthology.orchestra.run.smithi069.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T10:57:02.218 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T10:57:02.218 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T10:57:02.218 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T10:57:02.218 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T10:57:02.218 INFO:teuthology.orchestra.run.smithi069.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T10:57:02.218 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T10:57:02.218 INFO:teuthology.orchestra.run.smithi069.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T10:57:02.218 INFO:teuthology.orchestra.run.smithi069.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T10:57:02.218 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T10:57:02.218 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T10:57:02.218 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T10:57:02.218 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T10:57:02.218 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T10:57:02.219 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T10:57:02.219 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T10:57:02.219 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T10:57:02.219 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T10:57:02.219 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T10:57:02.219 INFO:teuthology.orchestra.run.smithi069.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T10:57:02.219 INFO:teuthology.orchestra.run.smithi069.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T10:57:02.219 INFO:teuthology.orchestra.run.smithi069.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T10:57:02.219 INFO:teuthology.orchestra.run.smithi069.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T10:57:02.219 INFO:teuthology.orchestra.run.smithi069.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T10:57:02.219 INFO:teuthology.orchestra.run.smithi069.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T10:57:02.219 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T10:57:02.219 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T10:57:02.219 INFO:teuthology.orchestra.run.smithi069.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T10:57:02.219 INFO:teuthology.orchestra.run.smithi069.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T10:57:02.220 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T10:57:02.220 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T10:57:02.220 INFO:teuthology.orchestra.run.smithi069.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T10:57:02.220 INFO:teuthology.orchestra.run.smithi069.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T10:57:02.220 INFO:teuthology.orchestra.run.smithi069.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T10:57:02.220 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T10:57:02.220 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T10:57:02.220 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T10:57:02.220 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T10:57:02.220 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T10:57:02.220 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T10:57:02.220 INFO:teuthology.orchestra.run.smithi069.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T10:57:02.220 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T10:57:02.220 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T10:57:02.220 INFO:teuthology.orchestra.run.smithi069.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T10:57:02.220 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T10:57:02.221 INFO:teuthology.orchestra.run.smithi069.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T10:57:02.221 INFO:teuthology.orchestra.run.smithi069.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T10:57:02.221 INFO:teuthology.orchestra.run.smithi069.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T10:57:02.221 INFO:teuthology.orchestra.run.smithi069.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T10:57:02.221 INFO:teuthology.orchestra.run.smithi069.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T10:57:02.221 INFO:teuthology.orchestra.run.smithi069.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T10:57:02.221 INFO:teuthology.orchestra.run.smithi069.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T10:57:02.221 INFO:teuthology.orchestra.run.smithi069.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T10:57:02.221 INFO:teuthology.orchestra.run.smithi069.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T10:57:02.221 INFO:teuthology.orchestra.run.smithi069.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T10:57:02.221 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T10:57:02.221 INFO:teuthology.orchestra.run.smithi069.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T10:57:02.221 INFO:teuthology.orchestra.run.smithi069.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T10:57:02.221 INFO:teuthology.orchestra.run.smithi069.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T10:57:02.221 INFO:teuthology.orchestra.run.smithi069.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T10:57:02.221 INFO:teuthology.orchestra.run.smithi069.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T10:57:02.222 INFO:teuthology.orchestra.run.smithi069.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T10:57:02.222 INFO:teuthology.orchestra.run.smithi069.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T10:57:02.222 INFO:teuthology.orchestra.run.smithi069.stdout:Installing weak dependencies: 2024-04-25T10:57:02.222 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T10:57:02.222 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:57:02.222 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2024-04-25T10:57:02.222 INFO:teuthology.orchestra.run.smithi069.stdout:======================================================================================= 2024-04-25T10:57:02.222 INFO:teuthology.orchestra.run.smithi069.stdout:Install 148 Packages 2024-04-25T10:57:02.222 INFO:teuthology.orchestra.run.smithi069.stdout:Upgrade 4 Packages 2024-04-25T10:57:02.222 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:57:02.223 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 214 M 2024-04-25T10:57:02.223 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2024-04-25T10:57:02.265 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-04-25T10:57:02.273 INFO:teuthology.orchestra.run.smithi146.stdout:======================================================================================= 2024-04-25T10:57:02.274 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repository Size 2024-04-25T10:57:02.274 INFO:teuthology.orchestra.run.smithi146.stdout:======================================================================================= 2024-04-25T10:57:02.274 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-04-25T10:57:02.274 INFO:teuthology.orchestra.run.smithi146.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T10:57:02.274 INFO:teuthology.orchestra.run.smithi146.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T10:57:02.274 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T10:57:02.274 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T10:57:02.274 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T10:57:02.274 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T10:57:02.274 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T10:57:02.274 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T10:57:02.274 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T10:57:02.274 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T10:57:02.275 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T10:57:02.275 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T10:57:02.275 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T10:57:02.275 INFO:teuthology.orchestra.run.smithi146.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T10:57:02.275 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T10:57:02.275 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T10:57:02.275 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T10:57:02.275 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T10:57:02.275 INFO:teuthology.orchestra.run.smithi146.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T10:57:02.275 INFO:teuthology.orchestra.run.smithi146.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T10:57:02.275 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T10:57:02.275 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T10:57:02.275 INFO:teuthology.orchestra.run.smithi146.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T10:57:02.275 INFO:teuthology.orchestra.run.smithi146.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T10:57:02.276 INFO:teuthology.orchestra.run.smithi146.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T10:57:02.276 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T10:57:02.276 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T10:57:02.276 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T10:57:02.276 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T10:57:02.276 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T10:57:02.276 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T10:57:02.276 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T10:57:02.276 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T10:57:02.276 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T10:57:02.276 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T10:57:02.276 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T10:57:02.276 INFO:teuthology.orchestra.run.smithi146.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T10:57:02.276 INFO:teuthology.orchestra.run.smithi146.stdout:Upgrading: 2024-04-25T10:57:02.276 INFO:teuthology.orchestra.run.smithi146.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T10:57:02.277 INFO:teuthology.orchestra.run.smithi146.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T10:57:02.277 INFO:teuthology.orchestra.run.smithi146.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T10:57:02.277 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T10:57:02.277 INFO:teuthology.orchestra.run.smithi146.stdout:Installing dependencies: 2024-04-25T10:57:02.277 INFO:teuthology.orchestra.run.smithi146.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T10:57:02.277 INFO:teuthology.orchestra.run.smithi146.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T10:57:02.277 INFO:teuthology.orchestra.run.smithi146.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T10:57:02.277 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T10:57:02.277 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T10:57:02.277 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T10:57:02.277 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T10:57:02.277 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T10:57:02.278 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T10:57:02.278 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T10:57:02.278 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T10:57:02.278 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T10:57:02.278 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T10:57:02.278 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T10:57:02.278 INFO:teuthology.orchestra.run.smithi146.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T10:57:02.278 INFO:teuthology.orchestra.run.smithi146.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T10:57:02.278 INFO:teuthology.orchestra.run.smithi146.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T10:57:02.278 INFO:teuthology.orchestra.run.smithi146.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T10:57:02.278 INFO:teuthology.orchestra.run.smithi146.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T10:57:02.278 INFO:teuthology.orchestra.run.smithi146.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T10:57:02.278 INFO:teuthology.orchestra.run.smithi146.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T10:57:02.279 INFO:teuthology.orchestra.run.smithi146.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T10:57:02.279 INFO:teuthology.orchestra.run.smithi146.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T10:57:02.279 INFO:teuthology.orchestra.run.smithi146.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T10:57:02.279 INFO:teuthology.orchestra.run.smithi146.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T10:57:02.279 INFO:teuthology.orchestra.run.smithi146.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T10:57:02.279 INFO:teuthology.orchestra.run.smithi146.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T10:57:02.279 INFO:teuthology.orchestra.run.smithi146.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T10:57:02.279 INFO:teuthology.orchestra.run.smithi146.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T10:57:02.279 INFO:teuthology.orchestra.run.smithi146.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T10:57:02.279 INFO:teuthology.orchestra.run.smithi146.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T10:57:02.279 INFO:teuthology.orchestra.run.smithi146.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T10:57:02.279 INFO:teuthology.orchestra.run.smithi146.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T10:57:02.279 INFO:teuthology.orchestra.run.smithi146.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T10:57:02.279 INFO:teuthology.orchestra.run.smithi146.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T10:57:02.280 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T10:57:02.280 INFO:teuthology.orchestra.run.smithi146.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T10:57:02.280 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T10:57:02.280 INFO:teuthology.orchestra.run.smithi146.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T10:57:02.280 INFO:teuthology.orchestra.run.smithi146.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T10:57:02.280 INFO:teuthology.orchestra.run.smithi146.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T10:57:02.280 INFO:teuthology.orchestra.run.smithi146.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T10:57:02.280 INFO:teuthology.orchestra.run.smithi146.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T10:57:02.280 INFO:teuthology.orchestra.run.smithi146.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T10:57:02.280 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T10:57:02.280 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T10:57:02.280 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T10:57:02.280 INFO:teuthology.orchestra.run.smithi146.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T10:57:02.280 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T10:57:02.281 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T10:57:02.281 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T10:57:02.281 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T10:57:02.281 INFO:teuthology.orchestra.run.smithi146.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T10:57:02.281 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T10:57:02.281 INFO:teuthology.orchestra.run.smithi146.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T10:57:02.281 INFO:teuthology.orchestra.run.smithi146.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T10:57:02.281 INFO:teuthology.orchestra.run.smithi146.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T10:57:02.281 INFO:teuthology.orchestra.run.smithi146.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T10:57:02.281 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T10:57:02.281 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T10:57:02.281 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T10:57:02.281 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T10:57:02.281 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T10:57:02.281 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T10:57:02.282 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T10:57:02.282 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T10:57:02.282 INFO:teuthology.orchestra.run.smithi146.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T10:57:02.282 INFO:teuthology.orchestra.run.smithi146.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T10:57:02.282 INFO:teuthology.orchestra.run.smithi146.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T10:57:02.282 INFO:teuthology.orchestra.run.smithi146.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T10:57:02.282 INFO:teuthology.orchestra.run.smithi146.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T10:57:02.282 INFO:teuthology.orchestra.run.smithi146.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T10:57:02.282 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T10:57:02.283 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T10:57:02.283 INFO:teuthology.orchestra.run.smithi146.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T10:57:02.283 INFO:teuthology.orchestra.run.smithi146.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T10:57:02.283 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T10:57:02.283 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T10:57:02.283 INFO:teuthology.orchestra.run.smithi146.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T10:57:02.283 INFO:teuthology.orchestra.run.smithi146.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T10:57:02.283 INFO:teuthology.orchestra.run.smithi146.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T10:57:02.283 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T10:57:02.283 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T10:57:02.283 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T10:57:02.283 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T10:57:02.284 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T10:57:02.284 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T10:57:02.284 INFO:teuthology.orchestra.run.smithi146.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T10:57:02.284 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T10:57:02.284 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T10:57:02.284 INFO:teuthology.orchestra.run.smithi146.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T10:57:02.284 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T10:57:02.284 INFO:teuthology.orchestra.run.smithi146.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T10:57:02.284 INFO:teuthology.orchestra.run.smithi146.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T10:57:02.284 INFO:teuthology.orchestra.run.smithi146.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T10:57:02.284 INFO:teuthology.orchestra.run.smithi146.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T10:57:02.284 INFO:teuthology.orchestra.run.smithi146.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T10:57:02.284 INFO:teuthology.orchestra.run.smithi146.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T10:57:02.284 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T10:57:02.284 INFO:teuthology.orchestra.run.smithi146.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T10:57:02.285 INFO:teuthology.orchestra.run.smithi146.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T10:57:02.285 INFO:teuthology.orchestra.run.smithi146.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T10:57:02.285 INFO:teuthology.orchestra.run.smithi146.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T10:57:02.285 INFO:teuthology.orchestra.run.smithi146.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T10:57:02.285 INFO:teuthology.orchestra.run.smithi146.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T10:57:02.285 INFO:teuthology.orchestra.run.smithi146.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T10:57:02.285 INFO:teuthology.orchestra.run.smithi146.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T10:57:02.285 INFO:teuthology.orchestra.run.smithi146.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T10:57:02.285 INFO:teuthology.orchestra.run.smithi146.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T10:57:02.285 INFO:teuthology.orchestra.run.smithi146.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T10:57:02.285 INFO:teuthology.orchestra.run.smithi146.stdout:Installing weak dependencies: 2024-04-25T10:57:02.285 INFO:teuthology.orchestra.run.smithi146.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T10:57:02.285 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:57:02.285 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-04-25T10:57:02.286 INFO:teuthology.orchestra.run.smithi146.stdout:======================================================================================= 2024-04-25T10:57:02.286 INFO:teuthology.orchestra.run.smithi146.stdout:Install 148 Packages 2024-04-25T10:57:02.286 INFO:teuthology.orchestra.run.smithi146.stdout:Upgrade 4 Packages 2024-04-25T10:57:02.286 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:57:02.286 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 214 M 2024-04-25T10:57:02.286 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-04-25T10:57:02.840 INFO:teuthology.orchestra.run.smithi069.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 31 kB/s | 6.6 kB 00:00 2024-04-25T10:57:03.157 INFO:teuthology.orchestra.run.smithi069.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 2.7 MB/s | 863 kB 00:00 2024-04-25T10:57:03.232 INFO:teuthology.orchestra.run.smithi069.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 1.9 MB/s | 147 kB 00:00 2024-04-25T10:57:03.535 INFO:teuthology.orchestra.run.smithi146.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-04-25T10:57:03.741 INFO:teuthology.orchestra.run.smithi069.stdout:(4/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 4.4 MB/s | 2.3 MB 00:00 2024-04-25T10:57:03.858 INFO:teuthology.orchestra.run.smithi069.stdout:(5/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 4.4 MB/s | 5.4 MB 00:01 2024-04-25T10:57:04.179 INFO:teuthology.orchestra.run.smithi069.stdout:(6/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 3.5 MB/s | 1.5 MB 00:00 2024-04-25T10:57:04.218 INFO:teuthology.orchestra.run.smithi146.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 1.2 MB/s | 863 kB 00:00 2024-04-25T10:57:04.293 INFO:teuthology.orchestra.run.smithi146.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 1.9 MB/s | 147 kB 00:00 2024-04-25T10:57:04.383 INFO:teuthology.orchestra.run.smithi069.stdout:(7/152): ceph-common-19.0.0-3291.g994e2f20.el9. 11 MB/s | 20 MB 00:01 2024-04-25T10:57:04.650 INFO:teuthology.orchestra.run.smithi069.stdout:(8/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 6.2 MB/s | 4.9 MB 00:00 2024-04-25T10:57:04.742 INFO:teuthology.orchestra.run.smithi069.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 275 kB/s | 25 kB 00:00 2024-04-25T10:57:04.860 INFO:teuthology.orchestra.run.smithi146.stdout:(4/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 3.2 MB/s | 5.4 MB 00:01 2024-04-25T10:57:05.277 INFO:teuthology.orchestra.run.smithi146.stdout:(5/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 2.3 MB/s | 2.3 MB 00:00 2024-04-25T10:57:05.444 INFO:teuthology.orchestra.run.smithi146.stdout:(6/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 2.6 MB/s | 1.5 MB 00:00 2024-04-25T10:57:05.601 INFO:teuthology.orchestra.run.smithi069.stdout:(10/152): ceph-radosgw-19.0.0-3291.g994e2f20.el 8.3 MB/s | 10 MB 00:01 2024-04-25T10:57:05.701 INFO:teuthology.orchestra.run.smithi069.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 2.2 MB/s | 224 kB 00:00 2024-04-25T10:57:05.785 INFO:teuthology.orchestra.run.smithi069.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 325 kB/s | 27 kB 00:00 2024-04-25T10:57:05.868 INFO:teuthology.orchestra.run.smithi069.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 381 kB/s | 32 kB 00:00 2024-04-25T10:57:06.094 INFO:teuthology.orchestra.run.smithi069.stdout:(14/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 3.3 MB/s | 756 kB 00:00 2024-04-25T10:57:06.169 INFO:teuthology.orchestra.run.smithi069.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 2.1 MB/s | 158 kB 00:00 2024-04-25T10:57:06.269 INFO:teuthology.orchestra.run.smithi069.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 1.2 MB/s | 127 kB 00:00 2024-04-25T10:57:06.411 INFO:teuthology.orchestra.run.smithi069.stdout:(17/152): libradosstriper1-19.0.0-3291.g994e2f2 3.4 MB/s | 497 kB 00:00 2024-04-25T10:57:07.353 INFO:teuthology.orchestra.run.smithi069.stdout:(18/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 5.5 MB/s | 5.2 MB 00:00 2024-04-25T10:57:07.436 INFO:teuthology.orchestra.run.smithi069.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 543 kB/s | 45 kB 00:00 2024-04-25T10:57:07.494 INFO:teuthology.orchestra.run.smithi146.stdout:(7/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 2.2 MB/s | 4.9 MB 00:02 2024-04-25T10:57:07.512 INFO:teuthology.orchestra.run.smithi069.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 1.8 MB/s | 135 kB 00:00 2024-04-25T10:57:07.595 INFO:teuthology.orchestra.run.smithi069.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 1.9 MB/s | 161 kB 00:00 2024-04-25T10:57:07.687 INFO:teuthology.orchestra.run.smithi069.stdout:(22/152): python3-rados-19.0.0-3291.g994e2f20.e 3.4 MB/s | 321 kB 00:00 2024-04-25T10:57:07.779 INFO:teuthology.orchestra.run.smithi069.stdout:(23/152): python3-rbd-19.0.0-3291.g994e2f20.el9 3.2 MB/s | 298 kB 00:00 2024-04-25T10:57:07.854 INFO:teuthology.orchestra.run.smithi069.stdout:(24/152): python3-rgw-19.0.0-3291.g994e2f20.el9 1.3 MB/s | 100 kB 00:00 2024-04-25T10:57:07.938 INFO:teuthology.orchestra.run.smithi069.stdout:(25/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 1.0 MB/s | 86 kB 00:00 2024-04-25T10:57:08.313 INFO:teuthology.orchestra.run.smithi069.stdout:(26/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 12 MB/s | 43 MB 00:03 2024-04-25T10:57:08.397 INFO:teuthology.orchestra.run.smithi069.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 2.0 MB/s | 169 kB 00:00 2024-04-25T10:57:08.448 INFO:teuthology.orchestra.run.smithi069.stdout:(28/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 6.0 MB/s | 3.0 MB 00:00 2024-04-25T10:57:08.464 INFO:teuthology.orchestra.run.smithi069.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3291.g 513 kB/s | 34 kB 00:00 2024-04-25T10:57:08.498 INFO:teuthology.orchestra.run.smithi069.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 2.8 MB/s | 142 kB 00:00 2024-04-25T10:57:08.823 INFO:teuthology.orchestra.run.smithi069.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 9.9 MB/s | 3.5 MB 00:00 2024-04-25T10:57:08.924 INFO:teuthology.orchestra.run.smithi069.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3291.g99 2.4 MB/s | 249 kB 00:00 2024-04-25T10:57:08.999 INFO:teuthology.orchestra.run.smithi069.stdout:(33/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 661 kB/s | 49 kB 00:00 2024-04-25T10:57:09.191 INFO:teuthology.orchestra.run.smithi069.stdout:(34/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 3.5 MB/s | 18 MB 00:05 2024-04-25T10:57:09.208 INFO:teuthology.orchestra.run.smithi069.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3291.g9 79 kB/s | 16 kB 00:00 2024-04-25T10:57:09.316 INFO:teuthology.orchestra.run.smithi069.stdout:(36/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 6.9 MB/s | 762 kB 00:00 2024-04-25T10:57:09.342 INFO:teuthology.orchestra.run.smithi069.stdout:(37/152): ceph-volume-19.0.0-3291.g994e2f20.el9 1.8 MB/s | 273 kB 00:00 2024-04-25T10:57:09.392 INFO:teuthology.orchestra.run.smithi069.stdout:(38/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 243 kB/s | 18 kB 00:00 2024-04-25T10:57:09.500 INFO:teuthology.orchestra.run.smithi069.stdout:(39/152): ceph-mgr-diskprediction-local-19.0.0- 7.4 MB/s | 7.4 MB 00:01 2024-04-25T10:57:09.626 INFO:teuthology.orchestra.run.smithi069.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 283 kB/s | 80 kB 00:00 2024-04-25T10:57:09.743 INFO:teuthology.orchestra.run.smithi069.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 807 kB/s | 195 kB 00:00 2024-04-25T10:57:09.768 INFO:teuthology.orchestra.run.smithi069.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-04-25T10:57:09.794 INFO:teuthology.orchestra.run.smithi146.stdout:(8/152): ceph-common-19.0.0-3291.g994e2f20.el9. 2.9 MB/s | 20 MB 00:06 2024-04-25T10:57:09.802 INFO:teuthology.orchestra.run.smithi069.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 803 kB 00:00 2024-04-25T10:57:09.827 INFO:teuthology.orchestra.run.smithi069.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-25T10:57:09.861 INFO:teuthology.orchestra.run.smithi069.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-25T10:57:09.886 INFO:teuthology.orchestra.run.smithi069.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-25T10:57:09.928 INFO:teuthology.orchestra.run.smithi069.stdout:(47/152): python3-cryptography-36.0.1-4.el9.x86 6.6 MB/s | 1.2 MB 00:00 2024-04-25T10:57:09.953 INFO:teuthology.orchestra.run.smithi069.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-04-25T10:57:09.977 INFO:teuthology.orchestra.run.smithi146.stdout:(9/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x86 3.9 MB/s | 18 MB 00:04 2024-04-25T10:57:09.994 INFO:teuthology.orchestra.run.smithi146.stdout:(10/152): ceph-selinux-19.0.0-3291.g994e2f20.el 126 kB/s | 25 kB 00:00 2024-04-25T10:57:09.995 INFO:teuthology.orchestra.run.smithi069.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-04-25T10:57:10.020 INFO:teuthology.orchestra.run.smithi069.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-25T10:57:10.054 INFO:teuthology.orchestra.run.smithi069.stdout:(51/152): automake-1.16.2-8.el9.noarch.rpm 4.0 MB/s | 689 kB 00:00 2024-04-25T10:57:10.079 INFO:teuthology.orchestra.run.smithi069.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 257 kB/s | 21 kB 00:00 2024-04-25T10:57:10.094 INFO:teuthology.orchestra.run.smithi146.stdout:(11/152): ceph-radosgw-19.0.0-3291.g994e2f20.el 3.9 MB/s | 10 MB 00:02 2024-04-25T10:57:10.104 INFO:teuthology.orchestra.run.smithi069.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 3.7 MB/s | 313 kB 00:00 2024-04-25T10:57:10.120 INFO:teuthology.orchestra.run.smithi146.stdout:(12/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 1.8 MB/s | 224 kB 00:00 2024-04-25T10:57:10.130 INFO:teuthology.orchestra.run.smithi069.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 433 kB/s | 33 kB 00:00 2024-04-25T10:57:10.145 INFO:teuthology.orchestra.run.smithi146.stdout:(13/152): cephfs-shell-19.0.0-3291.g994e2f20.el 542 kB/s | 27 kB 00:00 2024-04-25T10:57:10.170 INFO:teuthology.orchestra.run.smithi146.stdout:(14/152): libcephfs-devel-19.0.0-3291.g994e2f20 632 kB/s | 32 kB 00:00 2024-04-25T10:57:10.172 INFO:teuthology.orchestra.run.smithi069.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 272 kB/s | 18 kB 00:00 2024-04-25T10:57:10.197 INFO:teuthology.orchestra.run.smithi069.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 167 kB/s | 11 kB 00:00 2024-04-25T10:57:10.220 INFO:teuthology.orchestra.run.smithi146.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 3.1 MB/s | 158 kB 00:00 2024-04-25T10:57:10.230 INFO:teuthology.orchestra.run.smithi069.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-25T10:57:10.256 INFO:teuthology.orchestra.run.smithi069.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 772 kB/s | 45 kB 00:00 2024-04-25T10:57:10.271 INFO:teuthology.orchestra.run.smithi146.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 2.5 MB/s | 127 kB 00:00 2024-04-25T10:57:10.296 INFO:teuthology.orchestra.run.smithi146.stdout:(17/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 4.9 MB/s | 756 kB 00:00 2024-04-25T10:57:10.306 INFO:teuthology.orchestra.run.smithi069.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-04-25T10:57:10.339 INFO:teuthology.orchestra.run.smithi069.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 6.8 MB/s | 582 kB 00:00 2024-04-25T10:57:10.356 INFO:teuthology.orchestra.run.smithi069.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 506 kB/s | 25 kB 00:00 2024-04-25T10:57:10.363 INFO:teuthology.orchestra.run.smithi146.stdout:(18/152): libradosstriper1-19.0.0-3291.g994e2f2 5.3 MB/s | 497 kB 00:00 2024-04-25T10:57:10.390 INFO:teuthology.orchestra.run.smithi069.stdout:(62/152): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 662 kB 00:00 2024-04-25T10:57:10.414 INFO:teuthology.orchestra.run.smithi146.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 904 kB/s | 45 kB 00:00 2024-04-25T10:57:10.415 INFO:teuthology.orchestra.run.smithi069.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 629 kB/s | 47 kB 00:00 2024-04-25T10:57:10.440 INFO:teuthology.orchestra.run.smithi069.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-04-25T10:57:10.466 INFO:teuthology.orchestra.run.smithi069.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-04-25T10:57:10.472 INFO:teuthology.orchestra.run.smithi146.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 2.3 MB/s | 135 kB 00:00 2024-04-25T10:57:10.482 INFO:teuthology.orchestra.run.smithi069.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 525 kB/s | 35 kB 00:00 2024-04-25T10:57:10.531 INFO:teuthology.orchestra.run.smithi146.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 2.9 MB/s | 161 kB 00:00 2024-04-25T10:57:10.606 INFO:teuthology.orchestra.run.smithi146.stdout:(22/152): python3-rados-19.0.0-3291.g994e2f20.e 4.2 MB/s | 321 kB 00:00 2024-04-25T10:57:10.616 INFO:teuthology.orchestra.run.smithi069.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-25T10:57:10.681 INFO:teuthology.orchestra.run.smithi146.stdout:(23/152): python3-rbd-19.0.0-3291.g994e2f20.el9 3.9 MB/s | 298 kB 00:00 2024-04-25T10:57:10.708 INFO:teuthology.orchestra.run.smithi069.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 18 MB/s | 4.7 MB 00:00 2024-04-25T10:57:10.723 INFO:teuthology.orchestra.run.smithi146.stdout:(24/152): python3-rgw-19.0.0-3291.g994e2f20.el9 2.3 MB/s | 100 kB 00:00 2024-04-25T10:57:10.742 INFO:teuthology.orchestra.run.smithi069.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 7.1 MB/s | 911 kB 00:00 2024-04-25T10:57:10.767 INFO:teuthology.orchestra.run.smithi069.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-25T10:57:10.773 INFO:teuthology.orchestra.run.smithi146.stdout:(25/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 1.7 MB/s | 86 kB 00:00 2024-04-25T10:57:10.826 INFO:teuthology.orchestra.run.smithi069.stdout:(71/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-04-25T10:57:10.868 INFO:teuthology.orchestra.run.smithi069.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-04-25T10:57:10.893 INFO:teuthology.orchestra.run.smithi069.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-25T10:57:10.952 INFO:teuthology.orchestra.run.smithi069.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-04-25T10:57:11.027 INFO:teuthology.orchestra.run.smithi069.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-04-25T10:57:11.074 INFO:teuthology.orchestra.run.smithi146.stdout:(26/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 6.6 MB/s | 5.2 MB 00:00 2024-04-25T10:57:11.094 INFO:teuthology.orchestra.run.smithi069.stdout:(76/152): postgresql-server-13.14-1.el9.x86_64. 16 MB/s | 5.8 MB 00:00 2024-04-25T10:57:11.110 INFO:teuthology.orchestra.run.smithi069.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 307 kB/s | 26 kB 00:00 2024-04-25T10:57:11.132 INFO:teuthology.orchestra.run.smithi146.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 2.9 MB/s | 169 kB 00:00 2024-04-25T10:57:11.183 INFO:teuthology.orchestra.run.smithi146.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3291.g 686 kB/s | 34 kB 00:00 2024-04-25T10:57:11.186 INFO:teuthology.orchestra.run.smithi069.stdout:(78/152): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 6.0 MB 00:00 2024-04-25T10:57:11.203 INFO:teuthology.orchestra.run.smithi069.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 176 kB/s | 19 kB 00:00 2024-04-25T10:57:11.228 INFO:teuthology.orchestra.run.smithi069.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-04-25T10:57:11.241 INFO:teuthology.orchestra.run.smithi146.stdout:(29/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 6.5 MB/s | 3.0 MB 00:00 2024-04-25T10:57:11.245 INFO:teuthology.orchestra.run.smithi069.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 593 kB/s | 35 kB 00:00 2024-04-25T10:57:11.267 INFO:teuthology.orchestra.run.smithi146.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 1.7 MB/s | 142 kB 00:00 2024-04-25T10:57:11.287 INFO:teuthology.orchestra.run.smithi069.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-04-25T10:57:11.312 INFO:teuthology.orchestra.run.smithi069.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.2 MB/s | 222 kB 00:00 2024-04-25T10:57:11.346 INFO:teuthology.orchestra.run.smithi069.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-25T10:57:11.371 INFO:teuthology.orchestra.run.smithi069.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 721 kB/s | 42 kB 00:00 2024-04-25T10:57:11.405 INFO:teuthology.orchestra.run.smithi069.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-25T10:57:11.430 INFO:teuthology.orchestra.run.smithi069.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-04-25T10:57:11.497 INFO:teuthology.orchestra.run.smithi069.stdout:(88/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-04-25T10:57:11.522 INFO:teuthology.orchestra.run.smithi069.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 555 kB/s | 51 kB 00:00 2024-04-25T10:57:11.547 INFO:teuthology.orchestra.run.smithi069.stdout:(90/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.9 MB/s | 279 kB 00:00 2024-04-25T10:57:11.573 INFO:teuthology.orchestra.run.smithi069.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 711 kB/s | 54 kB 00:00 2024-04-25T10:57:11.606 INFO:teuthology.orchestra.run.smithi069.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 712 kB/s | 42 kB 00:00 2024-04-25T10:57:11.640 INFO:teuthology.orchestra.run.smithi069.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 663 kB/s | 44 kB 00:00 2024-04-25T10:57:11.665 INFO:teuthology.orchestra.run.smithi069.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-25T10:57:11.699 INFO:teuthology.orchestra.run.smithi069.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 612 kB/s | 36 kB 00:00 2024-04-25T10:57:11.700 INFO:teuthology.orchestra.run.smithi146.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 7.7 MB/s | 3.5 MB 00:00 2024-04-25T10:57:11.724 INFO:teuthology.orchestra.run.smithi069.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 808 kB/s | 47 kB 00:00 2024-04-25T10:57:11.766 INFO:teuthology.orchestra.run.smithi069.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.4 MB/s | 305 kB 00:00 2024-04-25T10:57:11.767 INFO:teuthology.orchestra.run.smithi146.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3291.g99 3.7 MB/s | 249 kB 00:00 2024-04-25T10:57:11.791 INFO:teuthology.orchestra.run.smithi069.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 866 kB/s | 58 kB 00:00 2024-04-25T10:57:11.817 INFO:teuthology.orchestra.run.smithi146.stdout:(33/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 987 kB/s | 49 kB 00:00 2024-04-25T10:57:11.824 INFO:teuthology.orchestra.run.smithi069.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-25T10:57:11.850 INFO:teuthology.orchestra.run.smithi069.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-25T10:57:11.868 INFO:teuthology.orchestra.run.smithi146.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3291.g9 326 kB/s | 16 kB 00:00 2024-04-25T10:57:11.892 INFO:teuthology.orchestra.run.smithi069.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 670 kB/s | 45 kB 00:00 2024-04-25T10:57:11.985 INFO:teuthology.orchestra.run.smithi146.stdout:(35/152): ceph-volume-19.0.0-3291.g994e2f20.el9 2.3 MB/s | 273 kB 00:00 2024-04-25T10:57:12.093 INFO:teuthology.orchestra.run.smithi069.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-04-25T10:57:12.118 INFO:teuthology.orchestra.run.smithi069.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 83 kB/s | 22 kB 00:00 2024-04-25T10:57:12.118 INFO:teuthology.orchestra.run.smithi146.stdout:(36/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 5.6 MB/s | 762 kB 00:00 2024-04-25T10:57:12.169 INFO:teuthology.orchestra.run.smithi146.stdout:(37/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 366 kB/s | 18 kB 00:00 2024-04-25T10:57:12.260 INFO:teuthology.orchestra.run.smithi069.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.3 MB/s | 863 kB 00:00 2024-04-25T10:57:12.286 INFO:teuthology.orchestra.run.smithi146.stdout:(38/152): ceph-mgr-diskprediction-local-19.0.0- 7.3 MB/s | 7.4 MB 00:01 2024-04-25T10:57:12.311 INFO:teuthology.orchestra.run.smithi146.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 567 kB/s | 80 kB 00:00 2024-04-25T10:57:12.370 INFO:teuthology.orchestra.run.smithi146.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2024-04-25T10:57:12.377 INFO:teuthology.orchestra.run.smithi069.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-04-25T10:57:12.402 INFO:teuthology.orchestra.run.smithi069.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 137 kB/s | 19 kB 00:00 2024-04-25T10:57:12.411 INFO:teuthology.orchestra.run.smithi146.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 6.3 MB/s | 803 kB 00:00 2024-04-25T10:57:12.435 INFO:teuthology.orchestra.run.smithi069.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.6 MB/s | 548 kB 00:00 2024-04-25T10:57:12.470 INFO:teuthology.orchestra.run.smithi146.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-04-25T10:57:12.477 INFO:teuthology.orchestra.run.smithi069.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 346 kB/s | 26 kB 00:00 2024-04-25T10:57:12.503 INFO:teuthology.orchestra.run.smithi069.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 726 kB/s | 49 kB 00:00 2024-04-25T10:57:12.545 INFO:teuthology.orchestra.run.smithi146.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 9.2 MB/s | 1.2 MB 00:00 2024-04-25T10:57:12.570 INFO:teuthology.orchestra.run.smithi069.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-04-25T10:57:12.570 INFO:teuthology.orchestra.run.smithi146.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-04-25T10:57:12.595 INFO:teuthology.orchestra.run.smithi069.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 575 kB/s | 67 kB 00:00 2024-04-25T10:57:12.629 INFO:teuthology.orchestra.run.smithi146.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-04-25T10:57:12.655 INFO:teuthology.orchestra.run.smithi146.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-04-25T10:57:12.662 INFO:teuthology.orchestra.run.smithi069.stdout:(112/152): libarrow-9.0.0-10.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-04-25T10:57:12.695 INFO:teuthology.orchestra.run.smithi069.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.5 MB/s | 838 kB 00:00 2024-04-25T10:57:12.721 INFO:teuthology.orchestra.run.smithi069.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 743 kB/s | 43 kB 00:00 2024-04-25T10:57:12.722 INFO:teuthology.orchestra.run.smithi146.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-04-25T10:57:12.746 INFO:teuthology.orchestra.run.smithi069.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 643 kB/s | 32 kB 00:00 2024-04-25T10:57:12.772 INFO:teuthology.orchestra.run.smithi146.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 5.8 MB/s | 689 kB 00:00 2024-04-25T10:57:12.779 INFO:teuthology.orchestra.run.smithi069.stdout:(116/152): python3-asyncssh-2.13.2-5.el9.noarch 2.9 MB/s | 548 kB 00:00 2024-04-25T10:57:12.796 INFO:teuthology.orchestra.run.smithi069.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-04-25T10:57:12.822 INFO:teuthology.orchestra.run.smithi069.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-25T10:57:12.839 INFO:teuthology.orchestra.run.smithi146.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-25T10:57:12.847 INFO:teuthology.orchestra.run.smithi069.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-04-25T10:57:12.864 INFO:teuthology.orchestra.run.smithi146.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 6.5 MB/s | 943 kB 00:00 2024-04-25T10:57:12.872 INFO:teuthology.orchestra.run.smithi069.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-04-25T10:57:12.889 INFO:teuthology.orchestra.run.smithi069.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 525 kB/s | 35 kB 00:00 2024-04-25T10:57:12.898 INFO:teuthology.orchestra.run.smithi146.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 368 kB/s | 21 kB 00:00 2024-04-25T10:57:12.914 INFO:teuthology.orchestra.run.smithi069.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 3.0 MB/s | 206 kB 00:00 2024-04-25T10:57:12.948 INFO:teuthology.orchestra.run.smithi146.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 3.7 MB/s | 313 kB 00:00 2024-04-25T10:57:12.956 INFO:teuthology.orchestra.run.smithi069.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-04-25T10:57:12.965 INFO:teuthology.orchestra.run.smithi146.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 488 kB/s | 33 kB 00:00 2024-04-25T10:57:12.973 INFO:teuthology.orchestra.run.smithi069.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 182 kB/s | 11 kB 00:00 2024-04-25T10:57:12.998 INFO:teuthology.orchestra.run.smithi069.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 423 kB/s | 18 kB 00:00 2024-04-25T10:57:13.007 INFO:teuthology.orchestra.run.smithi146.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 440 kB/s | 18 kB 00:00 2024-04-25T10:57:13.040 INFO:teuthology.orchestra.run.smithi069.stdout:(126/152): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-04-25T10:57:13.049 INFO:teuthology.orchestra.run.smithi146.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 267 kB/s | 11 kB 00:00 2024-04-25T10:57:13.057 INFO:teuthology.orchestra.run.smithi069.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 276 kB/s | 23 kB 00:00 2024-04-25T10:57:13.082 INFO:teuthology.orchestra.run.smithi069.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2024-04-25T10:57:13.100 INFO:teuthology.orchestra.run.smithi069.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 336 kB/s | 20 kB 00:00 2024-04-25T10:57:13.116 INFO:teuthology.orchestra.run.smithi146.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.4 MB/s | 160 kB 00:00 2024-04-25T10:57:13.150 INFO:teuthology.orchestra.run.smithi069.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 696 kB/s | 46 kB 00:00 2024-04-25T10:57:13.166 INFO:teuthology.orchestra.run.smithi146.stdout:(57/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-04-25T10:57:13.175 INFO:teuthology.orchestra.run.smithi069.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-04-25T10:57:13.208 INFO:teuthology.orchestra.run.smithi069.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 8.1 MB/s | 1.0 MB 00:00 2024-04-25T10:57:13.217 INFO:teuthology.orchestra.run.smithi146.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 452 kB/s | 45 kB 00:00 2024-04-25T10:57:13.225 INFO:teuthology.orchestra.run.smithi069.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 767 kB/s | 58 kB 00:00 2024-04-25T10:57:13.253 INFO:teuthology.orchestra.run.smithi069.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.4 MB/s | 272 kB 00:00 2024-04-25T10:57:13.275 INFO:teuthology.orchestra.run.smithi146.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-04-25T10:57:13.303 INFO:teuthology.orchestra.run.smithi069.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 174 kB/s | 16 kB 00:00 2024-04-25T10:57:13.317 INFO:teuthology.orchestra.run.smithi146.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 5.7 MB/s | 582 kB 00:00 2024-04-25T10:57:13.328 INFO:teuthology.orchestra.run.smithi069.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 877 kB/s | 90 kB 00:00 2024-04-25T10:57:13.342 INFO:teuthology.orchestra.run.smithi146.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 380 kB/s | 25 kB 00:00 2024-04-25T10:57:13.353 INFO:teuthology.orchestra.run.smithi069.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 281 kB/s | 28 kB 00:00 2024-04-25T10:57:13.379 INFO:teuthology.orchestra.run.smithi069.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-04-25T10:57:13.393 INFO:teuthology.orchestra.run.smithi146.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 631 kB/s | 47 kB 00:00 2024-04-25T10:57:13.404 INFO:teuthology.orchestra.run.smithi069.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-25T10:57:13.429 INFO:teuthology.orchestra.run.smithi069.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 784 kB/s | 59 kB 00:00 2024-04-25T10:57:13.443 INFO:teuthology.orchestra.run.smithi146.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.9 MB/s | 292 kB 00:00 2024-04-25T10:57:13.454 INFO:teuthology.orchestra.run.smithi069.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 473 kB/s | 36 kB 00:00 2024-04-25T10:57:13.480 INFO:teuthology.orchestra.run.smithi069.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-04-25T10:57:13.485 INFO:teuthology.orchestra.run.smithi146.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-04-25T10:57:13.505 INFO:teuthology.orchestra.run.smithi069.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-25T10:57:13.510 INFO:teuthology.orchestra.run.smithi146.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 528 kB/s | 35 kB 00:00 2024-04-25T10:57:13.530 INFO:teuthology.orchestra.run.smithi069.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-25T10:57:13.556 INFO:teuthology.orchestra.run.smithi069.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-04-25T10:57:13.581 INFO:teuthology.orchestra.run.smithi069.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-04-25T10:57:13.606 INFO:teuthology.orchestra.run.smithi069.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-25T10:57:13.648 INFO:teuthology.orchestra.run.smithi069.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-25T10:57:13.727 INFO:teuthology.orchestra.run.smithi146.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 20 MB/s | 4.7 MB 00:00 2024-04-25T10:57:13.794 INFO:teuthology.orchestra.run.smithi146.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-04-25T10:57:13.907 INFO:teuthology.orchestra.run.smithi069.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 5.2 MB/s | 1.3 MB 00:00 2024-04-25T10:57:13.911 INFO:teuthology.orchestra.run.smithi146.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 7.6 MB/s | 911 kB 00:00 2024-04-25T10:57:13.944 INFO:teuthology.orchestra.run.smithi146.stdout:(69/152): postgresql-13.14-1.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-04-25T10:57:14.011 INFO:teuthology.orchestra.run.smithi146.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.3 MB/s | 133 kB 00:00 2024-04-25T10:57:14.032 INFO:teuthology.orchestra.run.smithi069.stdout:(150/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 7.3 MB/s | 3.1 MB 00:00 2024-04-25T10:57:14.082 INFO:teuthology.orchestra.run.smithi069.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-04-25T10:57:14.120 INFO:teuthology.orchestra.run.smithi146.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-04-25T10:57:14.237 INFO:teuthology.orchestra.run.smithi146.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 20 MB/s | 5.8 MB 00:00 2024-04-25T10:57:14.262 INFO:teuthology.orchestra.run.smithi146.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 617 kB/s | 87 kB 00:00 2024-04-25T10:57:14.338 INFO:teuthology.orchestra.run.smithi146.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.2 MB/s | 244 kB 00:00 2024-04-25T10:57:14.357 INFO:teuthology.orchestra.run.smithi069.stdout:(152/152): librados2-19.0.0-3291.g994e2f20.el9. 4.3 MB/s | 3.3 MB 00:00 2024-04-25T10:57:14.360 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:57:14.360 INFO:teuthology.orchestra.run.smithi069.stdout:Total 18 MB/s | 214 MB 00:12 2024-04-25T10:57:14.413 INFO:teuthology.orchestra.run.smithi146.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-04-25T10:57:14.488 INFO:teuthology.orchestra.run.smithi146.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 340 kB/s | 26 kB 00:00 2024-04-25T10:57:14.564 INFO:teuthology.orchestra.run.smithi146.stdout:(77/152): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 6.0 MB 00:00 2024-04-25T10:57:14.614 INFO:teuthology.orchestra.run.smithi146.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 153 kB/s | 19 kB 00:00 2024-04-25T10:57:14.973 INFO:teuthology.orchestra.run.smithi146.stdout:(79/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 8.7 MB/s | 43 MB 00:04 2024-04-25T10:57:15.006 INFO:teuthology.orchestra.run.smithi146.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 389 kB/s | 172 kB 00:00 2024-04-25T10:57:15.031 INFO:teuthology.orchestra.run.smithi146.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 83 kB/s | 35 kB 00:00 2024-04-25T10:57:15.115 INFO:teuthology.orchestra.run.smithi146.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-04-25T10:57:15.149 INFO:teuthology.orchestra.run.smithi146.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.9 MB/s | 222 kB 00:00 2024-04-25T10:57:15.215 INFO:teuthology.orchestra.run.smithi146.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 772 kB/s | 77 kB 00:00 2024-04-25T10:57:15.241 INFO:teuthology.orchestra.run.smithi146.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 460 kB/s | 42 kB 00:00 2024-04-25T10:57:15.324 INFO:teuthology.orchestra.run.smithi146.stdout:(86/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-04-25T10:57:15.366 INFO:teuthology.orchestra.run.smithi146.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-04-25T10:57:15.392 INFO:teuthology.orchestra.run.smithi146.stdout:(88/152): python3-protobuf-3.14.0-13.el9.noarch 1.5 MB/s | 269 kB 00:00 2024-04-25T10:57:15.441 INFO:teuthology.orchestra.run.smithi146.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-04-25T10:57:15.467 INFO:teuthology.orchestra.run.smithi146.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 508 kB/s | 51 kB 00:00 2024-04-25T10:57:15.469 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2024-04-25T10:57:15.492 INFO:teuthology.orchestra.run.smithi146.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 535 kB/s | 54 kB 00:00 2024-04-25T10:57:15.543 INFO:teuthology.orchestra.run.smithi146.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 554 kB/s | 42 kB 00:00 2024-04-25T10:57:15.568 INFO:teuthology.orchestra.run.smithi146.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 588 kB/s | 44 kB 00:00 2024-04-25T10:57:15.611 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2024-04-25T10:57:15.611 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2024-04-25T10:57:15.660 INFO:teuthology.orchestra.run.smithi146.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 392 kB/s | 36 kB 00:00 2024-04-25T10:57:15.685 INFO:teuthology.orchestra.run.smithi146.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.5 MB/s | 221 kB 00:00 2024-04-25T10:57:15.735 INFO:teuthology.orchestra.run.smithi146.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 631 kB/s | 47 kB 00:00 2024-04-25T10:57:15.785 INFO:teuthology.orchestra.run.smithi146.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 305 kB 00:00 2024-04-25T10:57:15.811 INFO:teuthology.orchestra.run.smithi146.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 768 kB/s | 58 kB 00:00 2024-04-25T10:57:15.869 INFO:teuthology.orchestra.run.smithi146.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2024-04-25T10:57:15.894 INFO:teuthology.orchestra.run.smithi146.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 763 kB/s | 64 kB 00:00 2024-04-25T10:57:16.045 INFO:teuthology.orchestra.run.smithi146.stdout:(101/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-04-25T10:57:16.070 INFO:teuthology.orchestra.run.smithi146.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 223 kB/s | 45 kB 00:00 2024-04-25T10:57:16.095 INFO:teuthology.orchestra.run.smithi146.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 111 kB/s | 22 kB 00:00 2024-04-25T10:57:16.162 INFO:teuthology.orchestra.run.smithi146.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 7.2 MB/s | 863 kB 00:00 2024-04-25T10:57:16.546 INFO:teuthology.orchestra.run.smithi146.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 51 kB/s | 19 kB 00:00 2024-04-25T10:57:16.755 INFO:teuthology.orchestra.run.smithi146.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 469 kB/s | 309 kB 00:00 2024-04-25T10:57:16.847 INFO:teuthology.orchestra.run.smithi146.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 706 kB/s | 548 kB 00:00 2024-04-25T10:57:16.863 INFO:teuthology.orchestra.run.smithi146.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 239 kB/s | 26 kB 00:00 2024-04-25T10:57:16.947 INFO:teuthology.orchestra.run.smithi146.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 484 kB/s | 49 kB 00:00 2024-04-25T10:57:16.980 INFO:teuthology.orchestra.run.smithi146.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 577 kB/s | 67 kB 00:00 2024-04-25T10:57:17.065 INFO:teuthology.orchestra.run.smithi146.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-04-25T10:57:17.231 INFO:teuthology.orchestra.run.smithi146.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-04-25T10:57:17.265 INFO:teuthology.orchestra.run.smithi146.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.9 MB/s | 838 kB 00:00 2024-04-25T10:57:17.340 INFO:teuthology.orchestra.run.smithi146.stdout:(114/152): libarrow-9.0.0-10.el9.x86_64.rpm 5.6 MB/s | 4.4 MB 00:00 2024-04-25T10:57:17.343 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2024-04-25T10:57:17.343 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2024-04-25T10:57:17.365 INFO:teuthology.orchestra.run.smithi146.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 325 kB/s | 43 kB 00:00 2024-04-25T10:57:17.391 INFO:teuthology.orchestra.run.smithi146.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 256 kB/s | 32 kB 00:00 2024-04-25T10:57:17.449 INFO:teuthology.orchestra.run.smithi146.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-04-25T10:57:17.483 INFO:teuthology.orchestra.run.smithi146.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-04-25T10:57:17.516 INFO:teuthology.orchestra.run.smithi146.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-04-25T10:57:17.567 INFO:teuthology.orchestra.run.smithi146.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-04-25T10:57:17.592 INFO:teuthology.orchestra.run.smithi146.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 324 kB/s | 35 kB 00:00 2024-04-25T10:57:17.634 INFO:teuthology.orchestra.run.smithi146.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-04-25T10:57:17.718 INFO:teuthology.orchestra.run.smithi146.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-04-25T10:57:17.742 INFO:teuthology.orchestra.run.smithi146.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 98 kB/s | 11 kB 00:00 2024-04-25T10:57:17.768 INFO:teuthology.orchestra.run.smithi146.stdout:(125/152): python3-grpcio-tools-1.46.7-10.el9.x 822 kB/s | 144 kB 00:00 2024-04-25T10:57:17.827 INFO:teuthology.orchestra.run.smithi146.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-04-25T10:57:17.851 INFO:teuthology.orchestra.run.smithi146.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 213 kB/s | 23 kB 00:00 2024-04-25T10:57:17.877 INFO:teuthology.orchestra.run.smithi146.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 178 kB/s | 19 kB 00:00 2024-04-25T10:57:17.936 INFO:teuthology.orchestra.run.smithi146.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 181 kB/s | 20 kB 00:00 2024-04-25T10:57:17.986 INFO:teuthology.orchestra.run.smithi146.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-04-25T10:57:18.011 INFO:teuthology.orchestra.run.smithi146.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 346 kB/s | 46 kB 00:00 2024-04-25T10:57:18.045 INFO:teuthology.orchestra.run.smithi146.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 724 kB/s | 79 kB 00:00 2024-04-25T10:57:18.095 INFO:teuthology.orchestra.run.smithi146.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 531 kB/s | 58 kB 00:00 2024-04-25T10:57:18.137 INFO:teuthology.orchestra.run.smithi146.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-04-25T10:57:18.162 INFO:teuthology.orchestra.run.smithi146.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 140 kB/s | 16 kB 00:00 2024-04-25T10:57:18.204 INFO:teuthology.orchestra.run.smithi146.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 826 kB/s | 90 kB 00:00 2024-04-25T10:57:18.246 INFO:teuthology.orchestra.run.smithi146.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 260 kB/s | 28 kB 00:00 2024-04-25T10:57:18.280 INFO:teuthology.orchestra.run.smithi146.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 262 kB/s | 31 kB 00:00 2024-04-25T10:57:18.314 INFO:teuthology.orchestra.run.smithi146.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-25T10:57:18.356 INFO:teuthology.orchestra.run.smithi146.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 542 kB/s | 59 kB 00:00 2024-04-25T10:57:18.389 INFO:teuthology.orchestra.run.smithi146.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 329 kB/s | 36 kB 00:00 2024-04-25T10:57:18.423 INFO:teuthology.orchestra.run.smithi146.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 468 kB/s | 51 kB 00:00 2024-04-25T10:57:18.481 INFO:teuthology.orchestra.run.smithi146.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-04-25T10:57:18.498 INFO:teuthology.orchestra.run.smithi146.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 824 kB/s | 90 kB 00:00 2024-04-25T10:57:18.540 INFO:teuthology.orchestra.run.smithi146.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-04-25T10:57:18.590 INFO:teuthology.orchestra.run.smithi146.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 184 kB/s | 20 kB 00:00 2024-04-25T10:57:18.616 INFO:teuthology.orchestra.run.smithi146.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-04-25T10:57:18.674 INFO:teuthology.orchestra.run.smithi146.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-25T10:57:18.899 INFO:teuthology.orchestra.run.smithi146.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 6.0 MB/s | 1.3 MB 00:00 2024-04-25T10:57:18.975 INFO:teuthology.orchestra.run.smithi146.stdout:(150/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 8.6 MB/s | 3.1 MB 00:00 2024-04-25T10:57:19.066 INFO:teuthology.orchestra.run.smithi146.stdout:(151/152): librados2-19.0.0-3291.g994e2f20.el9. 7.0 MB/s | 3.3 MB 00:00 2024-04-25T10:57:19.250 INFO:teuthology.orchestra.run.smithi146.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.8 MB/s | 2.4 MB 00:00 2024-04-25T10:57:19.256 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:57:19.256 INFO:teuthology.orchestra.run.smithi146.stdout:Total 13 MB/s | 214 MB 00:16 2024-04-25T10:57:20.338 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-04-25T10:57:20.481 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-04-25T10:57:20.481 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-04-25T10:57:22.203 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-04-25T10:57:22.204 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-04-25T10:57:23.213 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2024-04-25T10:57:23.314 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T10:57:23.542 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T10:57:23.646 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T10:57:23.688 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T10:57:23.713 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T10:57:23.756 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T10:57:24.089 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T10:57:24.205 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:57:24.330 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:57:24.364 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:57:24.455 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:57:24.485 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T10:57:24.636 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T10:57:24.682 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T10:57:24.718 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:57:24.823 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:57:24.844 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T10:57:24.907 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T10:57:24.929 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T10:57:24.953 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T10:57:25.154 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T10:57:25.186 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T10:57:25.293 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T10:57:25.389 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T10:57:25.419 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T10:57:25.461 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T10:57:25.527 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T10:57:25.562 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T10:57:25.622 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T10:57:25.654 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T10:57:25.682 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T10:57:25.752 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T10:57:25.805 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T10:57:25.843 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T10:57:25.887 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T10:57:25.965 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T10:57:25.998 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T10:57:26.058 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T10:57:26.087 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T10:57:26.157 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T10:57:26.201 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T10:57:26.276 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T10:57:26.418 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T10:57:26.470 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T10:57:27.002 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T10:57:27.091 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T10:57:27.128 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T10:57:27.162 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T10:57:27.200 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T10:57:27.241 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T10:57:27.407 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T10:57:27.470 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T10:57:27.525 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T10:57:27.598 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T10:57:27.629 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T10:57:27.656 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T10:57:27.703 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T10:57:27.739 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T10:57:27.790 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T10:57:27.832 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T10:57:27.968 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T10:57:28.022 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T10:57:28.052 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T10:57:28.075 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-04-25T10:57:28.093 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T10:57:28.162 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T10:57:28.187 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T10:57:28.200 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T10:57:28.235 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T10:57:28.254 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T10:57:28.275 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T10:57:28.287 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T10:57:28.300 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T10:57:28.315 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T10:57:28.325 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T10:57:28.357 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T10:57:28.400 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T10:57:28.527 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T10:57:28.558 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T10:57:28.585 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T10:57:28.619 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T10:57:28.658 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T10:57:28.684 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T10:57:28.690 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T10:57:28.720 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T10:57:28.755 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T10:57:28.788 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T10:57:28.807 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:57:28.829 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T10:57:28.864 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T10:57:28.932 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:57:28.950 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T10:57:28.959 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:57:28.994 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T10:57:29.034 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T10:57:29.049 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:57:29.081 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T10:57:29.088 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T10:57:29.179 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T10:57:29.227 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T10:57:29.245 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T10:57:29.293 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T10:57:29.321 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:57:29.425 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:57:29.447 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T10:57:29.493 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T10:57:29.514 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T10:57:29.538 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T10:57:29.698 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T10:57:29.730 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T10:57:29.840 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T10:57:29.941 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T10:57:29.953 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T10:57:29.971 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T10:57:29.999 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T10:57:30.014 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T10:57:30.032 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T10:57:30.079 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T10:57:30.122 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T10:57:30.132 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T10:57:30.166 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T10:57:30.182 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T10:57:30.198 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T10:57:30.226 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T10:57:30.261 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T10:57:30.296 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T10:57:30.310 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T10:57:30.349 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T10:57:30.379 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T10:57:30.423 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T10:57:30.485 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T10:57:30.501 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T10:57:30.516 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T10:57:30.534 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T10:57:30.577 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T10:57:30.593 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T10:57:30.605 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T10:57:30.676 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T10:57:30.721 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T10:57:30.795 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T10:57:30.938 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T10:57:30.989 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T10:57:31.258 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T10:57:31.381 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T10:57:31.518 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T10:57:31.600 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T10:57:31.638 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T10:57:31.681 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T10:57:31.719 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T10:57:31.760 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T10:57:31.925 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T10:57:31.982 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T10:57:32.036 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T10:57:32.108 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T10:57:32.139 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T10:57:32.176 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T10:57:32.222 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T10:57:32.258 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T10:57:32.301 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T10:57:32.343 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T10:57:32.480 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T10:57:32.523 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T10:57:32.554 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T10:57:32.595 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T10:57:32.601 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T10:57:32.689 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T10:57:32.703 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T10:57:32.729 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T10:57:32.730 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T10:57:32.764 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T10:57:32.769 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T10:57:32.794 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T10:57:32.819 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T10:57:32.894 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T10:57:33.021 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T10:57:33.051 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T10:57:33.079 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T10:57:33.081 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T10:57:33.113 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T10:57:33.143 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T10:57:33.152 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T10:57:33.183 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T10:57:33.213 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T10:57:33.233 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T10:57:33.240 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T10:57:33.273 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T10:57:33.274 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T10:57:33.309 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T10:57:33.314 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T10:57:33.349 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T10:57:33.436 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T10:57:33.479 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T10:57:33.511 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T10:57:33.621 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T10:57:33.794 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T10:57:33.821 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T10:57:33.853 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T10:57:33.892 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T10:57:33.904 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T10:57:33.914 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T10:57:34.634 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T10:57:34.676 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T10:57:34.709 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T10:57:34.808 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T10:57:34.859 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T10:57:34.946 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T10:57:34.986 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T10:57:35.179 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T10:57:35.209 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T10:57:35.287 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T10:57:35.932 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T10:57:36.049 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T10:57:36.113 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:57:36.152 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:57:36.719 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:57:36.764 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T10:57:36.847 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T10:57:36.847 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T10:57:36.847 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T10:57:36.847 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:57:36.874 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T10:57:37.275 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T10:57:37.362 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T10:57:37.390 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T10:57:37.423 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T10:57:37.736 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T10:57:37.786 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T10:57:37.877 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T10:57:37.924 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T10:57:37.960 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T10:57:38.452 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T10:57:38.481 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T10:57:38.552 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T10:57:38.574 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T10:57:40.842 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:57:40.879 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:57:41.453 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:57:41.491 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T10:57:41.581 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T10:57:41.582 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T10:57:41.582 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T10:57:41.582 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:57:41.626 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T10:57:48.474 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T10:57:48.475 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /sys 2024-04-25T10:57:48.475 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /proc 2024-04-25T10:57:48.475 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /mnt 2024-04-25T10:57:48.475 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /var/tmp 2024-04-25T10:57:48.475 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /home 2024-04-25T10:57:48.475 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /root 2024-04-25T10:57:48.475 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /tmp 2024-04-25T10:57:48.475 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:57:48.652 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T10:57:48.667 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T10:57:48.667 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:57:48.667 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T10:57:48.668 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:57:50.247 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T10:57:50.272 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T10:57:50.272 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:57:50.272 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T10:57:50.273 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:57:50.273 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:57:50.273 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:57:50.497 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T10:57:50.520 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T10:57:50.520 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:57:50.520 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T10:57:50.520 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:57:50.520 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:57:50.520 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:57:50.981 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T10:57:51.004 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T10:57:51.004 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:57:51.004 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T10:57:51.004 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:57:51.004 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:57:51.005 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:57:51.128 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T10:57:51.185 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T10:57:51.241 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T10:57:51.428 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T10:57:51.474 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T10:57:51.552 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T10:57:51.641 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:57:51.774 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:57:51.815 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T10:57:51.847 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T10:57:51.874 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T10:57:52.335 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T10:57:52.365 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T10:57:52.397 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T10:57:52.459 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T10:57:52.496 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T10:57:53.174 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T10:57:53.174 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /sys 2024-04-25T10:57:53.174 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /proc 2024-04-25T10:57:53.174 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /mnt 2024-04-25T10:57:53.174 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /var/tmp 2024-04-25T10:57:53.175 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /home 2024-04-25T10:57:53.175 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /root 2024-04-25T10:57:53.175 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /tmp 2024-04-25T10:57:53.175 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:57:53.357 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T10:57:53.372 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T10:57:53.372 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:57:53.373 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T10:57:53.373 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:57:53.539 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T10:57:53.572 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T10:57:54.537 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T10:57:54.573 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T10:57:54.677 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T10:57:54.847 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T10:57:54.875 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T10:57:54.899 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T10:57:54.899 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:57:54.899 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T10:57:54.899 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:57:54.899 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:57:54.899 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:57:54.951 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T10:57:54.962 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T10:57:54.986 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T10:57:55.012 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T10:57:55.012 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:57:55.012 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T10:57:55.012 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:57:55.012 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:57:55.012 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:57:55.062 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T10:57:55.116 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T10:57:55.240 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T10:57:55.262 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T10:57:55.263 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:57:55.263 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T10:57:55.263 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:57:55.263 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:57:55.263 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:57:55.724 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T10:57:55.747 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T10:57:55.747 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:57:55.747 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T10:57:55.747 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:57:55.747 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:57:55.747 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:57:55.846 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T10:57:55.903 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T10:57:55.957 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T10:57:56.056 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T10:57:56.087 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T10:57:56.110 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T10:57:56.110 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:57:56.110 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T10:57:56.110 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:57:56.110 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:57:56.110 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:57:56.143 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T10:57:56.154 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T10:57:56.176 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T10:57:56.176 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:57:56.176 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T10:57:56.176 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:57:56.192 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T10:57:56.229 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T10:57:56.252 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T10:57:56.252 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:57:56.252 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T10:57:56.252 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:57:56.252 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:57:56.252 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:57:56.270 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T10:57:56.358 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:57:56.506 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:57:56.550 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T10:57:56.563 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T10:57:56.582 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T10:57:56.586 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T10:57:56.586 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:57:56.586 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T10:57:56.586 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:57:56.586 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:57:56.586 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:57:56.610 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T10:57:57.029 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T10:57:57.058 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T10:57:57.090 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T10:57:57.152 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T10:57:57.190 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T10:57:58.232 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T10:57:58.266 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T10:57:59.243 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T10:57:59.267 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T10:57:59.370 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T10:57:59.534 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T10:57:59.577 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T10:57:59.600 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T10:57:59.600 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:57:59.600 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T10:57:59.601 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:57:59.601 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:57:59.601 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:57:59.644 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T10:57:59.655 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T10:57:59.754 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T10:57:59.809 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T10:58:00.744 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T10:58:00.780 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T10:58:00.802 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T10:58:00.802 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:58:00.802 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T10:58:00.802 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:58:00.803 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:58:00.803 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:00.834 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T10:58:00.847 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T10:58:00.869 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T10:58:00.869 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:58:00.869 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T10:58:00.870 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:00.875 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T10:58:00.918 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T10:58:00.918 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T10:58:00.937 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T10:58:00.937 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:58:00.937 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T10:58:00.937 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:58:00.937 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:58:00.937 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:00.948 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T10:58:00.983 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T10:58:01.173 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T10:58:01.221 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T10:58:01.242 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T10:58:01.248 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T10:58:01.271 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T10:58:01.272 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:58:01.272 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T10:58:01.272 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:58:01.272 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:58:01.272 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:01.817 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:58:01.865 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:58:01.885 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:58:02.718 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T10:58:02.811 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T10:58:02.852 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T10:58:02.936 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T10:58:02.965 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T10:58:02.965 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:58:03.058 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:58:03.091 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:58:03.091 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:58:03.182 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:58:03.247 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:58:05.544 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T10:58:05.585 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T10:58:05.627 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T10:58:05.658 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T10:58:05.685 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T10:58:05.871 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T10:58:05.923 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T10:58:05.953 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T10:58:06.566 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:58:06.600 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:58:06.620 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:58:07.454 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T10:58:07.538 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T10:58:07.578 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T10:58:07.662 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T10:58:07.692 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T10:58:07.692 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:58:07.776 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:58:07.801 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:58:07.801 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:58:07.903 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:58:07.973 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:58:09.306 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:58:09.306 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T10:58:09.306 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T10:58:09.306 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T10:58:09.307 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T10:58:09.307 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T10:58:09.307 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T10:58:09.307 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T10:58:09.307 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:58:09.307 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:58:09.307 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T10:58:09.307 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T10:58:09.307 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T10:58:09.307 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:58:09.307 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T10:58:09.307 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T10:58:09.307 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T10:58:09.308 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T10:58:09.308 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T10:58:09.308 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T10:58:09.308 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T10:58:09.308 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T10:58:09.308 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T10:58:09.308 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T10:58:09.308 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T10:58:09.308 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T10:58:09.308 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T10:58:09.308 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T10:58:09.308 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T10:58:09.308 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T10:58:09.309 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T10:58:09.309 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T10:58:09.309 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T10:58:09.309 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T10:58:09.309 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T10:58:09.309 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T10:58:09.309 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T10:58:09.309 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T10:58:09.309 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T10:58:09.309 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T10:58:09.309 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T10:58:09.309 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T10:58:09.309 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T10:58:09.309 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T10:58:09.309 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T10:58:09.309 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T10:58:09.310 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T10:58:09.310 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T10:58:09.310 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T10:58:09.310 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T10:58:09.310 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T10:58:09.310 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T10:58:09.310 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T10:58:09.310 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T10:58:09.310 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T10:58:09.310 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T10:58:09.310 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T10:58:09.310 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T10:58:09.311 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T10:58:09.311 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T10:58:09.311 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T10:58:09.311 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T10:58:09.311 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T10:58:09.311 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T10:58:09.311 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T10:58:09.311 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T10:58:09.311 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T10:58:09.311 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T10:58:09.311 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T10:58:09.311 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T10:58:09.311 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T10:58:09.311 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T10:58:09.312 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T10:58:09.312 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T10:58:09.312 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T10:58:09.312 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T10:58:09.312 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T10:58:09.312 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T10:58:09.312 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T10:58:09.312 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T10:58:09.312 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T10:58:09.312 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T10:58:09.312 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T10:58:09.312 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T10:58:09.312 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T10:58:09.313 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T10:58:09.313 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T10:58:09.313 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T10:58:09.313 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T10:58:09.313 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T10:58:09.313 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T10:58:09.313 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T10:58:09.313 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T10:58:09.313 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T10:58:09.313 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T10:58:09.313 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T10:58:09.313 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T10:58:09.313 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T10:58:09.313 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T10:58:09.313 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T10:58:09.313 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T10:58:09.314 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T10:58:09.314 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T10:58:09.314 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T10:58:09.314 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T10:58:09.314 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T10:58:09.314 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T10:58:09.314 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T10:58:09.314 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T10:58:09.314 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T10:58:09.314 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T10:58:09.314 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T10:58:09.314 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T10:58:09.314 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T10:58:09.315 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T10:58:09.315 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T10:58:09.315 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T10:58:09.315 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T10:58:09.315 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:58:09.315 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:58:09.315 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T10:58:09.315 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T10:58:09.315 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T10:58:09.315 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T10:58:09.315 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T10:58:09.315 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T10:58:09.315 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T10:58:09.316 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T10:58:09.316 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T10:58:09.316 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T10:58:09.316 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T10:58:09.316 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T10:58:09.316 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T10:58:09.316 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T10:58:09.316 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T10:58:09.316 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T10:58:09.316 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T10:58:09.316 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T10:58:09.316 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T10:58:09.316 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T10:58:09.316 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T10:58:09.316 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T10:58:09.317 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T10:58:09.317 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T10:58:09.317 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T10:58:09.317 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T10:58:09.317 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T10:58:09.317 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T10:58:09.317 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T10:58:09.317 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T10:58:09.317 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T10:58:09.317 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T10:58:09.317 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T10:58:09.317 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T10:58:09.317 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:58:09.317 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T10:58:11.353 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:58:11.354 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:58:11.354 INFO:teuthology.orchestra.run.smithi069.stdout:Upgraded: 2024-04-25T10:58:11.354 INFO:teuthology.orchestra.run.smithi069.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:11.354 INFO:teuthology.orchestra.run.smithi069.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:11.354 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T10:58:11.354 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T10:58:11.354 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2024-04-25T10:58:11.354 INFO:teuthology.orchestra.run.smithi069.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T10:58:11.354 INFO:teuthology.orchestra.run.smithi069.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T10:58:11.354 INFO:teuthology.orchestra.run.smithi069.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T10:58:11.354 INFO:teuthology.orchestra.run.smithi069.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T10:58:11.354 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:11.354 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:11.355 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:11.355 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:11.355 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:58:11.355 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:11.355 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:11.355 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:11.355 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:58:11.355 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:58:11.355 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:58:11.355 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:58:11.355 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:58:11.355 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:11.356 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:11.356 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:58:11.356 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:11.356 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:11.356 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:11.356 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:58:11.356 INFO:teuthology.orchestra.run.smithi069.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:58:11.356 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:11.356 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:11.356 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:58:11.356 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T10:58:11.357 INFO:teuthology.orchestra.run.smithi069.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T10:58:11.357 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T10:58:11.357 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T10:58:11.357 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T10:58:11.357 INFO:teuthology.orchestra.run.smithi069.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T10:58:11.357 INFO:teuthology.orchestra.run.smithi069.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T10:58:11.357 INFO:teuthology.orchestra.run.smithi069.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T10:58:11.357 INFO:teuthology.orchestra.run.smithi069.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T10:58:11.357 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T10:58:11.357 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T10:58:11.357 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:11.357 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:11.357 INFO:teuthology.orchestra.run.smithi069.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:11.357 INFO:teuthology.orchestra.run.smithi069.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T10:58:11.358 INFO:teuthology.orchestra.run.smithi069.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T10:58:11.358 INFO:teuthology.orchestra.run.smithi069.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T10:58:11.358 INFO:teuthology.orchestra.run.smithi069.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T10:58:11.358 INFO:teuthology.orchestra.run.smithi069.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T10:58:11.358 INFO:teuthology.orchestra.run.smithi069.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:11.358 INFO:teuthology.orchestra.run.smithi069.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:11.358 INFO:teuthology.orchestra.run.smithi069.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T10:58:11.358 INFO:teuthology.orchestra.run.smithi069.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:11.358 INFO:teuthology.orchestra.run.smithi069.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T10:58:11.358 INFO:teuthology.orchestra.run.smithi069.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T10:58:11.358 INFO:teuthology.orchestra.run.smithi069.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T10:58:11.358 INFO:teuthology.orchestra.run.smithi069.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T10:58:11.359 INFO:teuthology.orchestra.run.smithi069.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T10:58:11.359 INFO:teuthology.orchestra.run.smithi069.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T10:58:11.359 INFO:teuthology.orchestra.run.smithi069.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T10:58:11.359 INFO:teuthology.orchestra.run.smithi069.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T10:58:11.359 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T10:58:11.359 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T10:58:11.359 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T10:58:11.359 INFO:teuthology.orchestra.run.smithi069.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T10:58:11.359 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T10:58:11.359 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T10:58:11.359 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T10:58:11.359 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T10:58:11.359 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T10:58:11.359 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T10:58:11.360 INFO:teuthology.orchestra.run.smithi069.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T10:58:11.360 INFO:teuthology.orchestra.run.smithi069.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T10:58:11.360 INFO:teuthology.orchestra.run.smithi069.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T10:58:11.360 INFO:teuthology.orchestra.run.smithi069.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T10:58:11.360 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T10:58:11.360 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:11.360 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:11.360 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:11.360 INFO:teuthology.orchestra.run.smithi069.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T10:58:11.360 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T10:58:11.360 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T10:58:11.360 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T10:58:11.360 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T10:58:11.360 INFO:teuthology.orchestra.run.smithi069.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T10:58:11.361 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T10:58:11.361 INFO:teuthology.orchestra.run.smithi069.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T10:58:11.361 INFO:teuthology.orchestra.run.smithi069.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T10:58:11.361 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T10:58:11.361 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T10:58:11.361 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T10:58:11.361 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T10:58:11.361 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T10:58:11.361 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T10:58:11.361 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T10:58:11.361 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T10:58:11.361 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T10:58:11.361 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T10:58:11.361 INFO:teuthology.orchestra.run.smithi069.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T10:58:11.362 INFO:teuthology.orchestra.run.smithi069.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T10:58:11.362 INFO:teuthology.orchestra.run.smithi069.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T10:58:11.362 INFO:teuthology.orchestra.run.smithi069.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T10:58:11.362 INFO:teuthology.orchestra.run.smithi069.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T10:58:11.362 INFO:teuthology.orchestra.run.smithi069.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T10:58:11.362 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T10:58:11.362 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T10:58:11.362 INFO:teuthology.orchestra.run.smithi069.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T10:58:11.362 INFO:teuthology.orchestra.run.smithi069.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T10:58:11.362 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T10:58:11.362 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T10:58:11.362 INFO:teuthology.orchestra.run.smithi069.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T10:58:11.362 INFO:teuthology.orchestra.run.smithi069.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T10:58:11.362 INFO:teuthology.orchestra.run.smithi069.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T10:58:11.363 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T10:58:11.363 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T10:58:11.363 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T10:58:11.363 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T10:58:11.363 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T10:58:11.363 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T10:58:11.363 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:11.364 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:11.364 INFO:teuthology.orchestra.run.smithi069.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T10:58:11.364 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T10:58:11.365 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T10:58:11.365 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:11.365 INFO:teuthology.orchestra.run.smithi069.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T10:58:11.365 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T10:58:11.365 INFO:teuthology.orchestra.run.smithi069.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T10:58:11.365 INFO:teuthology.orchestra.run.smithi069.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T10:58:11.365 INFO:teuthology.orchestra.run.smithi069.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T10:58:11.365 INFO:teuthology.orchestra.run.smithi069.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T10:58:11.365 INFO:teuthology.orchestra.run.smithi069.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T10:58:11.365 INFO:teuthology.orchestra.run.smithi069.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T10:58:11.365 INFO:teuthology.orchestra.run.smithi069.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T10:58:11.365 INFO:teuthology.orchestra.run.smithi069.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T10:58:11.365 INFO:teuthology.orchestra.run.smithi069.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T10:58:11.366 INFO:teuthology.orchestra.run.smithi069.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T10:58:11.366 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T10:58:11.366 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T10:58:11.366 INFO:teuthology.orchestra.run.smithi069.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T10:58:11.366 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:11.366 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:11.366 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:11.366 INFO:teuthology.orchestra.run.smithi069.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T10:58:11.366 INFO:teuthology.orchestra.run.smithi069.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T10:58:11.366 INFO:teuthology.orchestra.run.smithi069.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T10:58:11.366 INFO:teuthology.orchestra.run.smithi069.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T10:58:11.366 INFO:teuthology.orchestra.run.smithi069.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T10:58:11.366 INFO:teuthology.orchestra.run.smithi069.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T10:58:11.367 INFO:teuthology.orchestra.run.smithi069.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T10:58:11.367 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:58:11.367 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-04-25T10:58:11.776 DEBUG:teuthology.parallel:result is None 2024-04-25T10:58:13.826 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:58:13.826 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T10:58:13.826 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T10:58:13.827 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T10:58:13.827 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T10:58:13.827 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T10:58:13.827 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T10:58:13.827 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T10:58:13.827 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:58:13.827 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:58:13.827 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T10:58:13.827 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T10:58:13.827 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T10:58:13.827 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:58:13.827 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T10:58:13.827 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T10:58:13.827 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T10:58:13.828 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T10:58:13.828 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T10:58:13.828 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T10:58:13.828 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T10:58:13.828 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T10:58:13.828 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T10:58:13.828 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T10:58:13.828 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T10:58:13.828 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T10:58:13.828 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T10:58:13.828 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T10:58:13.828 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T10:58:13.829 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T10:58:13.829 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T10:58:13.829 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T10:58:13.829 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T10:58:13.829 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T10:58:13.829 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T10:58:13.829 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T10:58:13.829 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T10:58:13.829 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T10:58:13.829 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T10:58:13.829 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T10:58:13.829 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T10:58:13.829 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T10:58:13.829 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T10:58:13.830 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T10:58:13.830 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T10:58:13.830 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T10:58:13.830 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T10:58:13.830 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T10:58:13.830 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T10:58:13.830 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T10:58:13.830 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T10:58:13.830 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T10:58:13.830 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T10:58:13.830 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T10:58:13.831 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T10:58:13.831 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T10:58:13.831 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T10:58:13.831 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T10:58:13.831 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T10:58:13.831 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T10:58:13.831 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T10:58:13.831 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T10:58:13.831 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T10:58:13.831 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T10:58:13.831 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T10:58:13.831 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T10:58:13.831 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T10:58:13.832 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T10:58:13.832 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T10:58:13.832 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T10:58:13.832 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T10:58:13.832 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T10:58:13.832 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T10:58:13.832 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T10:58:13.832 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T10:58:13.832 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T10:58:13.832 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T10:58:13.832 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T10:58:13.832 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T10:58:13.832 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T10:58:13.832 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T10:58:13.833 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T10:58:13.833 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T10:58:13.833 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T10:58:13.833 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T10:58:13.833 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T10:58:13.833 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T10:58:13.833 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T10:58:13.833 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T10:58:13.833 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T10:58:13.833 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T10:58:13.833 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T10:58:13.833 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T10:58:13.833 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T10:58:13.833 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T10:58:13.834 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T10:58:13.834 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T10:58:13.834 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T10:58:13.834 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T10:58:13.834 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T10:58:13.834 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T10:58:13.834 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T10:58:13.834 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T10:58:13.834 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T10:58:13.834 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T10:58:13.834 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T10:58:13.834 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T10:58:13.835 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T10:58:13.835 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T10:58:13.835 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T10:58:13.835 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T10:58:13.835 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T10:58:13.835 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T10:58:13.835 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T10:58:13.835 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T10:58:13.835 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T10:58:13.835 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T10:58:13.835 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T10:58:13.836 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:58:13.836 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:58:13.836 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T10:58:13.836 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T10:58:13.836 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T10:58:13.836 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T10:58:13.836 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T10:58:13.836 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T10:58:13.836 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T10:58:13.836 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T10:58:13.836 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T10:58:13.836 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T10:58:13.836 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T10:58:13.837 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T10:58:13.837 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T10:58:13.837 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T10:58:13.837 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T10:58:13.837 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T10:58:13.837 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T10:58:13.837 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T10:58:13.837 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T10:58:13.837 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T10:58:13.837 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T10:58:13.837 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T10:58:13.837 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T10:58:13.837 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T10:58:13.838 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T10:58:13.838 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T10:58:13.838 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T10:58:13.838 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T10:58:13.838 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T10:58:13.838 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T10:58:13.838 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T10:58:13.838 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T10:58:13.838 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T10:58:13.838 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T10:58:13.838 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:58:13.838 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T10:58:15.972 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:58:15.972 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:15.972 INFO:teuthology.orchestra.run.smithi146.stdout:Upgraded: 2024-04-25T10:58:15.972 INFO:teuthology.orchestra.run.smithi146.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:15.972 INFO:teuthology.orchestra.run.smithi146.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:15.972 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T10:58:15.973 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T10:58:15.973 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-04-25T10:58:15.973 INFO:teuthology.orchestra.run.smithi146.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T10:58:15.973 INFO:teuthology.orchestra.run.smithi146.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T10:58:15.973 INFO:teuthology.orchestra.run.smithi146.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T10:58:15.973 INFO:teuthology.orchestra.run.smithi146.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T10:58:15.973 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:15.973 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:15.973 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:15.973 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:15.973 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:58:15.973 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:15.974 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:15.974 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:15.974 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:58:15.974 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:58:15.974 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:58:15.974 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:58:15.974 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:58:15.974 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:15.974 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:15.974 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:58:15.974 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:15.974 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:15.974 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:15.974 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:58:15.975 INFO:teuthology.orchestra.run.smithi146.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:58:15.975 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:15.975 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:15.975 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:58:15.975 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T10:58:15.975 INFO:teuthology.orchestra.run.smithi146.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T10:58:15.975 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T10:58:15.975 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T10:58:15.975 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T10:58:15.975 INFO:teuthology.orchestra.run.smithi146.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T10:58:15.975 INFO:teuthology.orchestra.run.smithi146.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T10:58:15.975 INFO:teuthology.orchestra.run.smithi146.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T10:58:15.975 INFO:teuthology.orchestra.run.smithi146.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T10:58:15.975 INFO:teuthology.orchestra.run.smithi146.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T10:58:15.976 INFO:teuthology.orchestra.run.smithi146.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T10:58:15.976 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:15.976 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:15.976 INFO:teuthology.orchestra.run.smithi146.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:15.976 INFO:teuthology.orchestra.run.smithi146.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T10:58:15.976 INFO:teuthology.orchestra.run.smithi146.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T10:58:15.976 INFO:teuthology.orchestra.run.smithi146.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T10:58:15.976 INFO:teuthology.orchestra.run.smithi146.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T10:58:15.976 INFO:teuthology.orchestra.run.smithi146.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T10:58:15.976 INFO:teuthology.orchestra.run.smithi146.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:15.976 INFO:teuthology.orchestra.run.smithi146.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:15.976 INFO:teuthology.orchestra.run.smithi146.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T10:58:15.976 INFO:teuthology.orchestra.run.smithi146.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:15.977 INFO:teuthology.orchestra.run.smithi146.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T10:58:15.977 INFO:teuthology.orchestra.run.smithi146.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T10:58:15.977 INFO:teuthology.orchestra.run.smithi146.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T10:58:15.977 INFO:teuthology.orchestra.run.smithi146.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T10:58:15.977 INFO:teuthology.orchestra.run.smithi146.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T10:58:15.977 INFO:teuthology.orchestra.run.smithi146.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T10:58:15.977 INFO:teuthology.orchestra.run.smithi146.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T10:58:15.977 INFO:teuthology.orchestra.run.smithi146.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T10:58:15.977 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T10:58:15.977 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T10:58:15.977 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T10:58:15.977 INFO:teuthology.orchestra.run.smithi146.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T10:58:15.977 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T10:58:15.977 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T10:58:15.977 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T10:58:15.978 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T10:58:15.978 INFO:teuthology.orchestra.run.smithi146.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T10:58:15.978 INFO:teuthology.orchestra.run.smithi146.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T10:58:15.978 INFO:teuthology.orchestra.run.smithi146.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T10:58:15.978 INFO:teuthology.orchestra.run.smithi146.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T10:58:15.978 INFO:teuthology.orchestra.run.smithi146.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T10:58:15.978 INFO:teuthology.orchestra.run.smithi146.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T10:58:15.978 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T10:58:15.978 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:15.978 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:15.978 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:15.978 INFO:teuthology.orchestra.run.smithi146.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T10:58:15.978 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T10:58:15.979 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T10:58:15.979 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T10:58:15.979 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T10:58:15.979 INFO:teuthology.orchestra.run.smithi146.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T10:58:15.979 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T10:58:15.979 INFO:teuthology.orchestra.run.smithi146.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T10:58:15.979 INFO:teuthology.orchestra.run.smithi146.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T10:58:15.979 INFO:teuthology.orchestra.run.smithi146.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T10:58:15.979 INFO:teuthology.orchestra.run.smithi146.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T10:58:15.979 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T10:58:15.979 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T10:58:15.979 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T10:58:15.979 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T10:58:15.979 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T10:58:15.979 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T10:58:15.980 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T10:58:15.980 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T10:58:15.980 INFO:teuthology.orchestra.run.smithi146.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T10:58:15.980 INFO:teuthology.orchestra.run.smithi146.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T10:58:15.980 INFO:teuthology.orchestra.run.smithi146.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T10:58:15.980 INFO:teuthology.orchestra.run.smithi146.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T10:58:15.980 INFO:teuthology.orchestra.run.smithi146.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T10:58:15.980 INFO:teuthology.orchestra.run.smithi146.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T10:58:15.980 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T10:58:15.980 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T10:58:15.980 INFO:teuthology.orchestra.run.smithi146.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T10:58:15.980 INFO:teuthology.orchestra.run.smithi146.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T10:58:15.980 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T10:58:15.980 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T10:58:15.980 INFO:teuthology.orchestra.run.smithi146.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T10:58:15.981 INFO:teuthology.orchestra.run.smithi146.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T10:58:15.981 INFO:teuthology.orchestra.run.smithi146.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T10:58:15.981 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T10:58:15.981 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T10:58:15.981 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T10:58:15.981 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T10:58:15.981 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T10:58:15.981 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T10:58:15.981 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:15.981 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:15.981 INFO:teuthology.orchestra.run.smithi146.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T10:58:15.981 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T10:58:15.981 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T10:58:15.981 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:15.982 INFO:teuthology.orchestra.run.smithi146.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T10:58:15.982 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T10:58:15.982 INFO:teuthology.orchestra.run.smithi146.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T10:58:15.982 INFO:teuthology.orchestra.run.smithi146.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T10:58:15.982 INFO:teuthology.orchestra.run.smithi146.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T10:58:15.982 INFO:teuthology.orchestra.run.smithi146.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T10:58:15.982 INFO:teuthology.orchestra.run.smithi146.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T10:58:15.982 INFO:teuthology.orchestra.run.smithi146.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T10:58:15.982 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T10:58:15.982 INFO:teuthology.orchestra.run.smithi146.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T10:58:15.982 INFO:teuthology.orchestra.run.smithi146.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T10:58:15.982 INFO:teuthology.orchestra.run.smithi146.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T10:58:15.982 INFO:teuthology.orchestra.run.smithi146.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T10:58:15.982 INFO:teuthology.orchestra.run.smithi146.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T10:58:15.983 INFO:teuthology.orchestra.run.smithi146.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T10:58:15.983 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:15.983 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:15.983 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:58:15.983 INFO:teuthology.orchestra.run.smithi146.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T10:58:15.983 INFO:teuthology.orchestra.run.smithi146.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T10:58:15.983 INFO:teuthology.orchestra.run.smithi146.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T10:58:15.983 INFO:teuthology.orchestra.run.smithi146.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T10:58:15.983 INFO:teuthology.orchestra.run.smithi146.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T10:58:15.983 INFO:teuthology.orchestra.run.smithi146.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T10:58:15.983 INFO:teuthology.orchestra.run.smithi146.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T10:58:15.984 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:15.984 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-04-25T10:58:16.393 DEBUG:teuthology.parallel:result is None 2024-04-25T10:58:16.394 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:58:16.541 DEBUG:teuthology.orchestra.run.smithi069:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T10:58:16.565 INFO:teuthology.orchestra.run.smithi069.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T10:58:16.566 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T10:58:16.566 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T10:58:16.568 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:58:16.778 DEBUG:teuthology.orchestra.run.smithi146:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T10:58:16.798 INFO:teuthology.orchestra.run.smithi146.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T10:58:16.798 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T10:58:16.799 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T10:58:16.800 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T10:58:16.801 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:58:16.801 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T10:58:16.831 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T10:58:16.831 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T10:58:16.864 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T10:58:16.864 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:58:16.864 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T10:58:16.898 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T10:58:16.964 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T10:58:16.964 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T10:58:16.989 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T10:58:17.052 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T10:58:17.052 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:58:17.052 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T10:58:17.078 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T10:58:17.144 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T10:58:17.144 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T10:58:17.167 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T10:58:17.232 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T10:58:17.232 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:58:17.232 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T10:58:17.257 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T10:58:17.322 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T10:58:17.322 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T10:58:17.346 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T10:58:17.411 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T10:58:17.499 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T10:58:17.499 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 777 /var/log/ceph 2024-04-25T10:58:17.502 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 777 /var/log/ceph 2024-04-25T10:58:17.526 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T10:58:17.526 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T10:58:17.567 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T10:58:17.590 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T10:58:17.590 DEBUG:teuthology.orchestra.run.smithi069:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T10:58:17.631 DEBUG:teuthology.orchestra.run.smithi146:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T10:58:17.662 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T10:58:17.662 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T10:58:17.663 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T10:58:17.663 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:58:17.663 DEBUG:teuthology.orchestra.run.smithi069:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T10:58:17.707 DEBUG:teuthology.orchestra.run.smithi069:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T10:58:17.816 DEBUG:teuthology.orchestra.run.smithi069:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T10:58:17.839 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T10:58:17.839 DEBUG:teuthology.orchestra.run.smithi146:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T10:58:17.853 DEBUG:teuthology.orchestra.run.smithi146:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T10:58:17.963 DEBUG:teuthology.orchestra.run.smithi146:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T10:58:17.992 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T10:58:17.992 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Replacing daemon mds.a'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T10:58:17.992 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673109', 'branch': 'wip-pdonnell-testing-20240425.015853-debug', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/cap-flush}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7673109', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240425.015853-debug'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Replacing daemon mds.a'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240425.015853-debug', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 85, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1880, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'sleep_before_teardown': 0, 'subset': '178/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240425.015853-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi069.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPVaRnKTw/F4gPOhcn3MJrgOv1TKo70DqBUsU44sG8l0vXWamAe8zF1gjRru8pRNCoutSKLlssRFrinEGkP+0u0=', 'smithi146.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDFfpZqoB7XKebiOPIa8TY4It0GGcnaZ4m6vSuJ68mo8RrxCBAwGSzawPnCWbGI1cP+kSU7K9iuysYNuXHZK008='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Replacing daemon mds.a'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_cap_flush']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_10:00:23', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T10:58:17.992 DEBUG:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T10:58:17.995 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T10:58:18.041 DEBUG:teuthology.orchestra.run.smithi069:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T10:58:18.049 DEBUG:teuthology.orchestra.run.smithi146:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T10:58:18.109 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:58:18.109 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T10:58:18.125 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T10:58:18.125 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_1 2024-04-25T10:58:18.180 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T10:58:18.180 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:58:18.180 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-25T10:58:18.180 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:58:18.180 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:58:18.180 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-04-25 10:58:03.663730499 +0000 2024-04-25T10:58:18.180 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-04-25 10:56:24.759241236 +0000 2024-04-25T10:58:18.180 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-04-25 10:56:24.759241236 +0000 2024-04-25T10:58:18.180 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-04-25 10:56:24.759241236 +0000 2024-04-25T10:58:18.181 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T10:58:18.242 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-04-25T10:58:18.242 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-04-25T10:58:18.242 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000205306 s, 2.5 MB/s 2024-04-25T10:58:18.243 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T10:58:18.299 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_2 2024-04-25T10:58:18.354 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T10:58:18.354 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:58:18.354 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-25T10:58:18.354 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:58:18.354 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:58:18.354 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-04-25 10:58:03.664730476 +0000 2024-04-25T10:58:18.354 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-04-25 10:56:25.101232914 +0000 2024-04-25T10:58:18.354 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-04-25 10:56:25.101232914 +0000 2024-04-25T10:58:18.355 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-04-25 10:56:25.101232914 +0000 2024-04-25T10:58:18.355 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T10:58:18.416 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-04-25T10:58:18.416 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-04-25T10:58:18.416 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000227524 s, 2.3 MB/s 2024-04-25T10:58:18.417 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T10:58:18.472 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_3 2024-04-25T10:58:18.526 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T10:58:18.527 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:58:18.527 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-25T10:58:18.527 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:58:18.527 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:58:18.527 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-04-25 10:58:03.664730476 +0000 2024-04-25T10:58:18.527 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-04-25 10:56:25.436224762 +0000 2024-04-25T10:58:18.527 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-04-25 10:56:25.436224762 +0000 2024-04-25T10:58:18.527 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-04-25 10:56:25.436224762 +0000 2024-04-25T10:58:18.527 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T10:58:18.588 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-04-25T10:58:18.588 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-04-25T10:58:18.588 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000204809 s, 2.5 MB/s 2024-04-25T10:58:18.589 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T10:58:18.644 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_4 2024-04-25T10:58:18.699 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T10:58:18.699 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:58:18.699 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-25T10:58:18.699 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:58:18.699 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:58:18.699 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-04-25 10:58:03.665730453 +0000 2024-04-25T10:58:18.699 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-04-25 10:56:25.777216465 +0000 2024-04-25T10:58:18.699 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-04-25 10:56:25.777216465 +0000 2024-04-25T10:58:18.699 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-04-25 10:56:25.777216465 +0000 2024-04-25T10:58:18.700 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T10:58:18.760 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-04-25T10:58:18.760 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-04-25T10:58:18.761 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000216366 s, 2.4 MB/s 2024-04-25T10:58:18.762 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T10:58:18.818 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:58:18.818 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T10:58:18.818 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T10:58:18.834 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T10:58:18.834 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_1 2024-04-25T10:58:18.889 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T10:58:18.889 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:58:18.889 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-25T10:58:18.890 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:58:18.890 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:58:18.890 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-04-25 10:58:08.325472710 +0000 2024-04-25T10:58:18.890 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-04-25 10:56:22.621919586 +0000 2024-04-25T10:58:18.890 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-04-25 10:56:22.621919586 +0000 2024-04-25T10:58:18.890 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-04-25 10:56:22.621919586 +0000 2024-04-25T10:58:18.890 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T10:58:18.952 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-04-25T10:58:18.953 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-04-25T10:58:18.953 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000212376 s, 2.4 MB/s 2024-04-25T10:58:18.954 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T10:58:19.011 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_2 2024-04-25T10:58:19.066 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T10:58:19.066 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:58:19.067 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-25T10:58:19.067 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:58:19.067 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:58:19.067 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-04-25 10:58:08.325472710 +0000 2024-04-25T10:58:19.067 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-04-25 10:56:22.980913870 +0000 2024-04-25T10:58:19.067 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-04-25 10:56:22.980913870 +0000 2024-04-25T10:58:19.067 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-04-25 10:56:22.980913870 +0000 2024-04-25T10:58:19.067 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T10:58:19.128 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-04-25T10:58:19.128 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-04-25T10:58:19.129 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000196159 s, 2.6 MB/s 2024-04-25T10:58:19.130 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T10:58:19.186 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_3 2024-04-25T10:58:19.242 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T10:58:19.242 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:58:19.242 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-25T10:58:19.242 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:58:19.242 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:58:19.242 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-04-25 10:58:08.326472695 +0000 2024-04-25T10:58:19.242 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-04-25 10:56:23.324908391 +0000 2024-04-25T10:58:19.242 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-04-25 10:56:23.324908391 +0000 2024-04-25T10:58:19.242 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-04-25 10:56:23.324908391 +0000 2024-04-25T10:58:19.243 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T10:58:19.304 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-04-25T10:58:19.304 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-04-25T10:58:19.304 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000213716 s, 2.4 MB/s 2024-04-25T10:58:19.305 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T10:58:19.361 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_4 2024-04-25T10:58:19.416 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T10:58:19.416 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:58:19.416 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-25T10:58:19.416 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:58:19.416 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:58:19.416 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-04-25 10:58:08.326472695 +0000 2024-04-25T10:58:19.416 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-04-25 10:56:23.656903104 +0000 2024-04-25T10:58:19.416 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-04-25 10:56:23.656903104 +0000 2024-04-25T10:58:19.416 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-04-25 10:56:23.656903104 +0000 2024-04-25T10:58:19.417 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T10:58:19.479 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-04-25T10:58:19.480 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-04-25T10:58:19.480 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000265079 s, 1.9 MB/s 2024-04-25T10:58:19.481 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T10:58:19.536 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:58:19.536 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi069.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi146.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T10:58:19.536 INFO:tasks.ceph:Generating config... 2024-04-25T10:58:19.538 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T10:58:19.538 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T10:58:19.538 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T10:58:19.538 INFO:tasks.ceph:[client] pid file = 2024-04-25T10:58:19.538 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T10:58:19.538 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T10:58:19.538 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-25T10:58:19.538 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T10:58:19.538 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T10:58:19.538 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T10:58:19.538 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T10:58:19.538 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T10:58:19.538 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T10:58:19.539 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T10:58:19.539 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T10:58:19.539 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T10:58:19.539 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T10:58:19.539 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-25T10:58:19.539 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T10:58:19.539 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T10:58:19.539 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T10:58:19.539 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T10:58:19.539 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T10:58:19.539 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-25T10:58:19.539 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T10:58:19.539 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-25T10:58:19.539 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-25T10:58:19.539 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-25T10:58:19.539 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T10:58:19.540 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T10:58:19.540 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-25T10:58:19.540 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-25T10:58:19.540 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T10:58:19.540 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T10:58:19.540 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-25T10:58:19.540 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T10:58:19.540 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T10:58:19.540 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T10:58:19.540 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T10:58:19.540 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T10:58:19.540 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T10:58:19.540 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T10:58:19.541 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T10:58:19.601 INFO:teuthology.orchestra.run.smithi069.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T10:58:19.604 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T10:58:19.656 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:58:19.679 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.69'), ('mon.b', '172.21.15.146'), ('mon.c', '[v2:172.21.15.146:3301,v1:172.21.15.146:6790]')] 2024-04-25T10:58:19.679 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.69,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T10:58:19.680 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:58:19.681 DEBUG:teuthology.orchestra.run.smithi069:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T10:58:19.734 DEBUG:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.69 --add b 172.21.15.146 --addv c '[v2:172.21.15.146:3301,v1:172.21.15.146:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:58:19.829 INFO:teuthology.orchestra.run.smithi069.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:58:19.829 INFO:teuthology.orchestra.run.smithi069.stdout:monmaptool: generated fsid ed0c8fe4-1bbb-4705-be72-eeea5bef859e 2024-04-25T10:58:19.829 INFO:teuthology.orchestra.run.smithi069.stdout:setting min_mon_release = quincy 2024-04-25T10:58:19.829 INFO:teuthology.orchestra.run.smithi069.stdout:epoch 0 2024-04-25T10:58:19.829 INFO:teuthology.orchestra.run.smithi069.stdout:fsid ed0c8fe4-1bbb-4705-be72-eeea5bef859e 2024-04-25T10:58:19.829 INFO:teuthology.orchestra.run.smithi069.stdout:last_changed 2024-04-25T10:58:19.829516+0000 2024-04-25T10:58:19.829 INFO:teuthology.orchestra.run.smithi069.stdout:created 2024-04-25T10:58:19.829516+0000 2024-04-25T10:58:19.829 INFO:teuthology.orchestra.run.smithi069.stdout:min_mon_release 17 (quincy) 2024-04-25T10:58:19.829 INFO:teuthology.orchestra.run.smithi069.stdout:election_strategy: 1 2024-04-25T10:58:19.829 INFO:teuthology.orchestra.run.smithi069.stdout:0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-04-25T10:58:19.830 INFO:teuthology.orchestra.run.smithi069.stdout:1: [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] mon.b 2024-04-25T10:58:19.830 INFO:teuthology.orchestra.run.smithi069.stdout:2: [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] mon.c 2024-04-25T10:58:19.830 INFO:teuthology.orchestra.run.smithi069.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T10:58:19.831 DEBUG:teuthology.orchestra.run.smithi069:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T10:58:19.845 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID ed0c8fe4-1bbb-4705-be72-eeea5bef859e... 2024-04-25T10:58:19.846 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:58:19.887 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:58:19.925 INFO:teuthology.orchestra.run.smithi069.stdout:[global] 2024-04-25T10:58:19.925 INFO:teuthology.orchestra.run.smithi069.stdout: chdir = "" 2024-04-25T10:58:19.925 INFO:teuthology.orchestra.run.smithi069.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:58:19.925 INFO:teuthology.orchestra.run.smithi069.stdout: auth supported = cephx 2024-04-25T10:58:19.925 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:58:19.925 INFO:teuthology.orchestra.run.smithi069.stdout: filestore xattr use omap = true 2024-04-25T10:58:19.925 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:58:19.925 INFO:teuthology.orchestra.run.smithi069.stdout: mon clock drift allowed = 1.000 2024-04-25T10:58:19.925 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:58:19.925 INFO:teuthology.orchestra.run.smithi069.stdout: osd crush chooseleaf type = 0 2024-04-25T10:58:19.926 INFO:teuthology.orchestra.run.smithi069.stdout: auth debug = true 2024-04-25T10:58:19.926 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:58:19.926 INFO:teuthology.orchestra.run.smithi069.stdout: ms die on old message = true 2024-04-25T10:58:19.926 INFO:teuthology.orchestra.run.smithi069.stdout: ms die on bug = true 2024-04-25T10:58:19.926 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:58:19.926 INFO:teuthology.orchestra.run.smithi069.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:58:19.926 INFO:teuthology.orchestra.run.smithi069.stdout: mon pg warn max object skew = 0 2024-04-25T10:58:19.926 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:58:19.926 INFO:teuthology.orchestra.run.smithi069.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:58:19.926 INFO:teuthology.orchestra.run.smithi069.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:58:19.926 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:58:19.926 INFO:teuthology.orchestra.run.smithi069.stdout: osd pool default size = 2 2024-04-25T10:58:19.926 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:58:19.927 INFO:teuthology.orchestra.run.smithi069.stdout: mon osd allow primary affinity = true 2024-04-25T10:58:19.927 INFO:teuthology.orchestra.run.smithi069.stdout: mon osd allow pg remap = true 2024-04-25T10:58:19.927 INFO:teuthology.orchestra.run.smithi069.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:58:19.927 INFO:teuthology.orchestra.run.smithi069.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:58:19.927 INFO:teuthology.orchestra.run.smithi069.stdout: mon warn on no sortbitwise = false 2024-04-25T10:58:19.927 INFO:teuthology.orchestra.run.smithi069.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:58:19.927 INFO:teuthology.orchestra.run.smithi069.stdout: mon warn on too few osds = false 2024-04-25T10:58:19.927 INFO:teuthology.orchestra.run.smithi069.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:58:19.927 INFO:teuthology.orchestra.run.smithi069.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:58:19.927 INFO:teuthology.orchestra.run.smithi069.stdout: mon_allow_pool_size_one = true 2024-04-25T10:58:19.927 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:58:19.927 INFO:teuthology.orchestra.run.smithi069.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:58:19.927 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:58:19.927 INFO:teuthology.orchestra.run.smithi069.stdout: osd default data pool replay window = 5 2024-04-25T10:58:19.928 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:58:19.928 INFO:teuthology.orchestra.run.smithi069.stdout: mon allow pool delete = true 2024-04-25T10:58:19.928 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:58:19.928 INFO:teuthology.orchestra.run.smithi069.stdout: mon cluster log file level = debug 2024-04-25T10:58:19.928 INFO:teuthology.orchestra.run.smithi069.stdout: debug asserts on shutdown = true 2024-04-25T10:58:19.928 INFO:teuthology.orchestra.run.smithi069.stdout: mon health detail to clog = false 2024-04-25T10:58:19.928 INFO:teuthology.orchestra.run.smithi069.stdout: mon host = "172.21.15.69,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2024-04-25T10:58:19.928 INFO:teuthology.orchestra.run.smithi069.stdout: ms die on skipped message = False 2024-04-25T10:58:19.928 INFO:teuthology.orchestra.run.smithi069.stdout: fsid = ed0c8fe4-1bbb-4705-be72-eeea5bef859e 2024-04-25T10:58:19.928 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:58:19.928 INFO:teuthology.orchestra.run.smithi069.stdout:[osd] 2024-04-25T10:58:19.928 INFO:teuthology.orchestra.run.smithi069.stdout: osd journal size = 100 2024-04-25T10:58:19.928 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:58:19.928 INFO:teuthology.orchestra.run.smithi069.stdout: osd scrub load threshold = 5.0 2024-04-25T10:58:19.929 INFO:teuthology.orchestra.run.smithi069.stdout: osd scrub max interval = 600 2024-04-25T10:58:19.929 INFO:teuthology.orchestra.run.smithi069.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:58:19.929 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:58:19.929 INFO:teuthology.orchestra.run.smithi069.stdout: osd recover clone overlap = true 2024-04-25T10:58:19.929 INFO:teuthology.orchestra.run.smithi069.stdout: osd recovery max chunk = 1048576 2024-04-25T10:58:19.929 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:58:19.929 INFO:teuthology.orchestra.run.smithi069.stdout: osd debug shutdown = true 2024-04-25T10:58:19.929 INFO:teuthology.orchestra.run.smithi069.stdout: osd debug op order = true 2024-04-25T10:58:19.929 INFO:teuthology.orchestra.run.smithi069.stdout: osd debug verify stray on activate = true 2024-04-25T10:58:19.929 INFO:teuthology.orchestra.run.smithi069.stdout: osd debug trim objects = true 2024-04-25T10:58:19.929 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:58:19.929 INFO:teuthology.orchestra.run.smithi069.stdout: osd open classes on start = true 2024-04-25T10:58:19.929 INFO:teuthology.orchestra.run.smithi069.stdout: osd debug pg log writeout = true 2024-04-25T10:58:19.929 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:58:19.930 INFO:teuthology.orchestra.run.smithi069.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:58:19.930 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:58:19.930 INFO:teuthology.orchestra.run.smithi069.stdout: osd map max advance = 10 2024-04-25T10:58:19.930 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:58:19.930 INFO:teuthology.orchestra.run.smithi069.stdout: journal zero on create = true 2024-04-25T10:58:19.930 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:58:19.930 INFO:teuthology.orchestra.run.smithi069.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:58:19.930 INFO:teuthology.orchestra.run.smithi069.stdout: filestore apply finisher threads = 3 2024-04-25T10:58:19.930 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:58:19.930 INFO:teuthology.orchestra.run.smithi069.stdout: bdev debug aio = true 2024-04-25T10:58:19.930 INFO:teuthology.orchestra.run.smithi069.stdout: osd debug misdirected ops = true 2024-04-25T10:58:19.930 INFO:teuthology.orchestra.run.smithi069.stdout: bdev async discard = True 2024-04-25T10:58:19.930 INFO:teuthology.orchestra.run.smithi069.stdout: bdev enable discard = True 2024-04-25T10:58:19.930 INFO:teuthology.orchestra.run.smithi069.stdout: bluestore allocator = bitmap 2024-04-25T10:58:19.931 INFO:teuthology.orchestra.run.smithi069.stdout: bluestore block size = 96636764160 2024-04-25T10:58:19.931 INFO:teuthology.orchestra.run.smithi069.stdout: bluestore fsck on mount = True 2024-04-25T10:58:19.931 INFO:teuthology.orchestra.run.smithi069.stdout: debug bluefs = 1/20 2024-04-25T10:58:19.931 INFO:teuthology.orchestra.run.smithi069.stdout: debug bluestore = 1/20 2024-04-25T10:58:19.931 INFO:teuthology.orchestra.run.smithi069.stdout: debug ms = 1 2024-04-25T10:58:19.931 INFO:teuthology.orchestra.run.smithi069.stdout: debug osd = 20 2024-04-25T10:58:19.931 INFO:teuthology.orchestra.run.smithi069.stdout: debug rocksdb = 4/10 2024-04-25T10:58:19.931 INFO:teuthology.orchestra.run.smithi069.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:58:19.931 INFO:teuthology.orchestra.run.smithi069.stdout: mon osd full ratio = 0.9 2024-04-25T10:58:19.931 INFO:teuthology.orchestra.run.smithi069.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:58:19.931 INFO:teuthology.orchestra.run.smithi069.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:58:19.931 INFO:teuthology.orchestra.run.smithi069.stdout: osd objectstore = bluestore 2024-04-25T10:58:19.931 INFO:teuthology.orchestra.run.smithi069.stdout: osd op complaint time = 180 2024-04-25T10:58:19.931 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:58:19.932 INFO:teuthology.orchestra.run.smithi069.stdout:[mgr] 2024-04-25T10:58:19.932 INFO:teuthology.orchestra.run.smithi069.stdout: debug ms = 1 2024-04-25T10:58:19.932 INFO:teuthology.orchestra.run.smithi069.stdout: debug mgr = 20 2024-04-25T10:58:19.932 INFO:teuthology.orchestra.run.smithi069.stdout: debug mon = 20 2024-04-25T10:58:19.932 INFO:teuthology.orchestra.run.smithi069.stdout: debug auth = 20 2024-04-25T10:58:19.932 INFO:teuthology.orchestra.run.smithi069.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:58:19.932 INFO:teuthology.orchestra.run.smithi069.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:58:19.932 INFO:teuthology.orchestra.run.smithi069.stdout: mgr/telemetry/nag = false 2024-04-25T10:58:19.932 INFO:teuthology.orchestra.run.smithi069.stdout: debug client = 20 2024-04-25T10:58:19.932 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:58:19.932 INFO:teuthology.orchestra.run.smithi069.stdout:[mon] 2024-04-25T10:58:19.932 INFO:teuthology.orchestra.run.smithi069.stdout: debug ms = 1 2024-04-25T10:58:19.932 INFO:teuthology.orchestra.run.smithi069.stdout: debug mon = 20 2024-04-25T10:58:19.932 INFO:teuthology.orchestra.run.smithi069.stdout: debug paxos = 20 2024-04-25T10:58:19.933 INFO:teuthology.orchestra.run.smithi069.stdout: debug auth = 20 2024-04-25T10:58:19.933 INFO:teuthology.orchestra.run.smithi069.stdout: mon data avail warn = 5 2024-04-25T10:58:19.933 INFO:teuthology.orchestra.run.smithi069.stdout: mon mgr mkfs grace = 240 2024-04-25T10:58:19.933 INFO:teuthology.orchestra.run.smithi069.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:58:19.933 INFO:teuthology.orchestra.run.smithi069.stdout: mon osd reporter subtree level = osd 2024-04-25T10:58:19.933 INFO:teuthology.orchestra.run.smithi069.stdout: mon osd prime pg temp = true 2024-04-25T10:58:19.933 INFO:teuthology.orchestra.run.smithi069.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:58:19.933 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:58:19.933 INFO:teuthology.orchestra.run.smithi069.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:58:19.933 INFO:teuthology.orchestra.run.smithi069.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:58:19.933 INFO:teuthology.orchestra.run.smithi069.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:58:19.933 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:58:19.933 INFO:teuthology.orchestra.run.smithi069.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:58:19.933 INFO:teuthology.orchestra.run.smithi069.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:58:19.933 INFO:teuthology.orchestra.run.smithi069.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:58:19.933 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:58:19.934 INFO:teuthology.orchestra.run.smithi069.stdout: # 1m isn't quite enough 2024-04-25T10:58:19.934 INFO:teuthology.orchestra.run.smithi069.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:58:19.934 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:58:19.934 INFO:teuthology.orchestra.run.smithi069.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:58:19.934 INFO:teuthology.orchestra.run.smithi069.stdout: mon down mkfs grace = 300 2024-04-25T10:58:19.934 INFO:teuthology.orchestra.run.smithi069.stdout: mon op complaint time = 120 2024-04-25T10:58:19.934 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:58:19.934 INFO:teuthology.orchestra.run.smithi069.stdout:[client] 2024-04-25T10:58:19.934 INFO:teuthology.orchestra.run.smithi069.stdout: rgw cache enabled = true 2024-04-25T10:58:19.934 INFO:teuthology.orchestra.run.smithi069.stdout: rgw enable ops log = true 2024-04-25T10:58:19.934 INFO:teuthology.orchestra.run.smithi069.stdout: rgw enable usage log = true 2024-04-25T10:58:19.934 INFO:teuthology.orchestra.run.smithi069.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:58:19.934 INFO:teuthology.orchestra.run.smithi069.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:58:19.934 INFO:teuthology.orchestra.run.smithi069.stdout: client mount timeout = 600 2024-04-25T10:58:19.935 INFO:teuthology.orchestra.run.smithi069.stdout: debug client = 20 2024-04-25T10:58:19.935 INFO:teuthology.orchestra.run.smithi069.stdout: debug ms = 1 2024-04-25T10:58:19.935 INFO:teuthology.orchestra.run.smithi069.stdout: pid file = "" 2024-04-25T10:58:19.935 INFO:teuthology.orchestra.run.smithi069.stdout: rados mon op timeout = 900 2024-04-25T10:58:19.935 INFO:teuthology.orchestra.run.smithi069.stdout: rados osd op timeout = 900 2024-04-25T10:58:19.935 INFO:teuthology.orchestra.run.smithi069.stdout:[mon.a] 2024-04-25T10:58:19.935 INFO:teuthology.orchestra.run.smithi069.stdout:[mon.b] 2024-04-25T10:58:19.935 INFO:teuthology.orchestra.run.smithi069.stdout:[mon.c] 2024-04-25T10:58:19.935 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.a] 2024-04-25T10:58:19.935 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.c] 2024-04-25T10:58:19.935 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.b] 2024-04-25T10:58:19.935 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.d] 2024-04-25T10:58:19.935 INFO:teuthology.orchestra.run.smithi069.stdout:[mds] 2024-04-25T10:58:19.935 INFO:teuthology.orchestra.run.smithi069.stdout: debug mds = 20 2024-04-25T10:58:19.935 INFO:teuthology.orchestra.run.smithi069.stdout: debug mds balancer = 20 2024-04-25T10:58:19.936 INFO:teuthology.orchestra.run.smithi069.stdout: debug ms = 1 2024-04-25T10:58:19.936 INFO:teuthology.orchestra.run.smithi069.stdout: mds debug frag = True 2024-04-25T10:58:19.936 INFO:teuthology.orchestra.run.smithi069.stdout: mds debug scatterstat = True 2024-04-25T10:58:19.936 INFO:teuthology.orchestra.run.smithi069.stdout: mds op complaint time = 180 2024-04-25T10:58:19.936 INFO:teuthology.orchestra.run.smithi069.stdout: mds verify scatter = True 2024-04-25T10:58:19.936 INFO:teuthology.orchestra.run.smithi069.stdout: osd op complaint time = 180 2024-04-25T10:58:19.936 INFO:teuthology.orchestra.run.smithi069.stdout: rados mon op timeout = 900 2024-04-25T10:58:19.936 INFO:teuthology.orchestra.run.smithi069.stdout: rados osd op timeout = 900 2024-04-25T10:58:19.936 INFO:teuthology.orchestra.run.smithi146.stdout:[global] 2024-04-25T10:58:19.937 INFO:teuthology.orchestra.run.smithi146.stdout: chdir = "" 2024-04-25T10:58:19.937 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:58:19.937 INFO:teuthology.orchestra.run.smithi146.stdout: auth supported = cephx 2024-04-25T10:58:19.937 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:19.937 INFO:teuthology.orchestra.run.smithi146.stdout: filestore xattr use omap = true 2024-04-25T10:58:19.937 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:19.937 INFO:teuthology.orchestra.run.smithi146.stdout: mon clock drift allowed = 1.000 2024-04-25T10:58:19.937 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:19.937 INFO:teuthology.orchestra.run.smithi146.stdout: osd crush chooseleaf type = 0 2024-04-25T10:58:19.937 INFO:teuthology.orchestra.run.smithi146.stdout: auth debug = true 2024-04-25T10:58:19.937 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:19.937 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on old message = true 2024-04-25T10:58:19.937 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on bug = true 2024-04-25T10:58:19.938 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:19.938 INFO:teuthology.orchestra.run.smithi146.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:58:19.938 INFO:teuthology.orchestra.run.smithi146.stdout: mon pg warn max object skew = 0 2024-04-25T10:58:19.938 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:19.938 INFO:teuthology.orchestra.run.smithi146.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:58:19.938 INFO:teuthology.orchestra.run.smithi146.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:58:19.938 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:19.938 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default size = 2 2024-04-25T10:58:19.938 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:19.938 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow primary affinity = true 2024-04-25T10:58:19.938 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow pg remap = true 2024-04-25T10:58:19.938 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:58:19.938 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:58:19.939 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on no sortbitwise = false 2024-04-25T10:58:19.939 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:58:19.939 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on too few osds = false 2024-04-25T10:58:19.939 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:58:19.939 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:58:19.939 INFO:teuthology.orchestra.run.smithi146.stdout: mon_allow_pool_size_one = true 2024-04-25T10:58:19.939 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:19.939 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:58:19.939 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:19.939 INFO:teuthology.orchestra.run.smithi146.stdout: osd default data pool replay window = 5 2024-04-25T10:58:19.939 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:19.939 INFO:teuthology.orchestra.run.smithi146.stdout: mon allow pool delete = true 2024-04-25T10:58:19.939 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:19.940 INFO:teuthology.orchestra.run.smithi146.stdout: mon cluster log file level = debug 2024-04-25T10:58:19.940 INFO:teuthology.orchestra.run.smithi146.stdout: debug asserts on shutdown = true 2024-04-25T10:58:19.940 INFO:teuthology.orchestra.run.smithi146.stdout: mon health detail to clog = false 2024-04-25T10:58:19.940 INFO:teuthology.orchestra.run.smithi146.stdout: mon host = "172.21.15.69,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2024-04-25T10:58:19.940 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on skipped message = False 2024-04-25T10:58:19.940 INFO:teuthology.orchestra.run.smithi146.stdout: fsid = ed0c8fe4-1bbb-4705-be72-eeea5bef859e 2024-04-25T10:58:19.940 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:19.940 INFO:teuthology.orchestra.run.smithi146.stdout:[osd] 2024-04-25T10:58:19.940 INFO:teuthology.orchestra.run.smithi146.stdout: osd journal size = 100 2024-04-25T10:58:19.940 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:19.940 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub load threshold = 5.0 2024-04-25T10:58:19.940 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub max interval = 600 2024-04-25T10:58:19.940 INFO:teuthology.orchestra.run.smithi146.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:58:19.940 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:19.940 INFO:teuthology.orchestra.run.smithi146.stdout: osd recover clone overlap = true 2024-04-25T10:58:19.941 INFO:teuthology.orchestra.run.smithi146.stdout: osd recovery max chunk = 1048576 2024-04-25T10:58:19.941 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:19.941 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug shutdown = true 2024-04-25T10:58:19.941 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug op order = true 2024-04-25T10:58:19.941 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug verify stray on activate = true 2024-04-25T10:58:19.941 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug trim objects = true 2024-04-25T10:58:19.941 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:19.941 INFO:teuthology.orchestra.run.smithi146.stdout: osd open classes on start = true 2024-04-25T10:58:19.941 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug pg log writeout = true 2024-04-25T10:58:19.941 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:19.941 INFO:teuthology.orchestra.run.smithi146.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:58:19.941 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:19.941 INFO:teuthology.orchestra.run.smithi146.stdout: osd map max advance = 10 2024-04-25T10:58:19.942 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:19.942 INFO:teuthology.orchestra.run.smithi146.stdout: journal zero on create = true 2024-04-25T10:58:19.942 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:19.942 INFO:teuthology.orchestra.run.smithi146.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:58:19.942 INFO:teuthology.orchestra.run.smithi146.stdout: filestore apply finisher threads = 3 2024-04-25T10:58:19.942 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:19.942 INFO:teuthology.orchestra.run.smithi146.stdout: bdev debug aio = true 2024-04-25T10:58:19.942 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug misdirected ops = true 2024-04-25T10:58:19.942 INFO:teuthology.orchestra.run.smithi146.stdout: bdev async discard = True 2024-04-25T10:58:19.942 INFO:teuthology.orchestra.run.smithi146.stdout: bdev enable discard = True 2024-04-25T10:58:19.942 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore allocator = bitmap 2024-04-25T10:58:19.942 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore block size = 96636764160 2024-04-25T10:58:19.942 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore fsck on mount = True 2024-04-25T10:58:19.942 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluefs = 1/20 2024-04-25T10:58:19.942 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluestore = 1/20 2024-04-25T10:58:19.943 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-04-25T10:58:19.943 INFO:teuthology.orchestra.run.smithi146.stdout: debug osd = 20 2024-04-25T10:58:19.943 INFO:teuthology.orchestra.run.smithi146.stdout: debug rocksdb = 4/10 2024-04-25T10:58:19.943 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:58:19.943 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd full ratio = 0.9 2024-04-25T10:58:19.943 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:58:19.943 INFO:teuthology.orchestra.run.smithi146.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:58:19.943 INFO:teuthology.orchestra.run.smithi146.stdout: osd objectstore = bluestore 2024-04-25T10:58:19.943 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2024-04-25T10:58:19.943 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:19.943 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr] 2024-04-25T10:58:19.943 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-04-25T10:58:19.943 INFO:teuthology.orchestra.run.smithi146.stdout: debug mgr = 20 2024-04-25T10:58:19.943 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2024-04-25T10:58:19.944 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2024-04-25T10:58:19.944 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:58:19.944 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:58:19.944 INFO:teuthology.orchestra.run.smithi146.stdout: mgr/telemetry/nag = false 2024-04-25T10:58:19.944 INFO:teuthology.orchestra.run.smithi146.stdout: debug client = 20 2024-04-25T10:58:19.944 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:19.944 INFO:teuthology.orchestra.run.smithi146.stdout:[mon] 2024-04-25T10:58:19.944 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-04-25T10:58:19.944 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2024-04-25T10:58:19.944 INFO:teuthology.orchestra.run.smithi146.stdout: debug paxos = 20 2024-04-25T10:58:19.944 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2024-04-25T10:58:19.944 INFO:teuthology.orchestra.run.smithi146.stdout: mon data avail warn = 5 2024-04-25T10:58:19.944 INFO:teuthology.orchestra.run.smithi146.stdout: mon mgr mkfs grace = 240 2024-04-25T10:58:19.944 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:58:19.944 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd reporter subtree level = osd 2024-04-25T10:58:19.945 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd prime pg temp = true 2024-04-25T10:58:19.945 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:58:19.945 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:19.945 INFO:teuthology.orchestra.run.smithi146.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:58:19.945 INFO:teuthology.orchestra.run.smithi146.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:58:19.945 INFO:teuthology.orchestra.run.smithi146.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:58:19.945 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:19.945 INFO:teuthology.orchestra.run.smithi146.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:58:19.945 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:58:19.945 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:58:19.945 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:19.945 INFO:teuthology.orchestra.run.smithi146.stdout: # 1m isn't quite enough 2024-04-25T10:58:19.945 INFO:teuthology.orchestra.run.smithi146.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:58:19.945 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:19.945 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:58:19.946 INFO:teuthology.orchestra.run.smithi146.stdout: mon down mkfs grace = 300 2024-04-25T10:58:19.946 INFO:teuthology.orchestra.run.smithi146.stdout: mon op complaint time = 120 2024-04-25T10:58:19.946 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T10:58:19.946 INFO:teuthology.orchestra.run.smithi146.stdout:[client] 2024-04-25T10:58:19.946 INFO:teuthology.orchestra.run.smithi146.stdout: rgw cache enabled = true 2024-04-25T10:58:19.946 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable ops log = true 2024-04-25T10:58:19.946 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable usage log = true 2024-04-25T10:58:19.946 INFO:teuthology.orchestra.run.smithi146.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:58:19.946 INFO:teuthology.orchestra.run.smithi146.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:58:19.946 INFO:teuthology.orchestra.run.smithi146.stdout: client mount timeout = 600 2024-04-25T10:58:19.946 INFO:teuthology.orchestra.run.smithi146.stdout: debug client = 20 2024-04-25T10:58:19.946 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-04-25T10:58:19.946 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = "" 2024-04-25T10:58:19.946 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2024-04-25T10:58:19.946 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2024-04-25T10:58:19.947 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.a] 2024-04-25T10:58:19.947 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.b] 2024-04-25T10:58:19.947 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.c] 2024-04-25T10:58:19.949 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.a] 2024-04-25T10:58:19.949 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.c] 2024-04-25T10:58:19.949 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.b] 2024-04-25T10:58:19.949 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.d] 2024-04-25T10:58:19.949 INFO:teuthology.orchestra.run.smithi146.stdout:[mds] 2024-04-25T10:58:19.949 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds = 20 2024-04-25T10:58:19.949 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds balancer = 20 2024-04-25T10:58:19.949 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-04-25T10:58:19.949 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug frag = True 2024-04-25T10:58:19.949 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug scatterstat = True 2024-04-25T10:58:19.950 INFO:teuthology.orchestra.run.smithi146.stdout: mds op complaint time = 180 2024-04-25T10:58:19.950 INFO:teuthology.orchestra.run.smithi146.stdout: mds verify scatter = True 2024-04-25T10:58:19.950 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2024-04-25T10:58:19.950 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2024-04-25T10:58:19.950 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2024-04-25T10:58:19.951 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T10:58:19.951 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T10:58:20.053 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T10:58:20.053 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:58:20.053 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T10:58:20.068 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:58:20.068 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T10:58:20.121 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi069.front.sepia.ceph.com 2024-04-25T10:58:20.122 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:58:20.122 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T10:58:20.122 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:58:20.193 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:58:20.193 DEBUG:teuthology.orchestra.run.smithi069:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:58:20.248 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi146.front.sepia.ceph.com 2024-04-25T10:58:20.248 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T10:58:20.248 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T10:58:20.248 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:58:20.281 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T10:58:20.282 DEBUG:teuthology.orchestra.run.smithi146:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:58:20.336 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T10:58:20.336 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T10:58:20.336 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T10:58:20.393 INFO:teuthology.orchestra.run.smithi069.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T10:58:20.396 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T10:58:20.470 INFO:teuthology.orchestra.run.smithi146.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T10:58:20.473 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T10:58:20.473 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T10:58:20.530 INFO:teuthology.orchestra.run.smithi069.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T10:58:20.533 DEBUG:teuthology.orchestra.run.smithi069:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T10:58:20.557 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T10:58:20.654 INFO:teuthology.orchestra.run.smithi069.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T10:58:20.657 DEBUG:teuthology.orchestra.run.smithi069:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T10:58:20.680 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T10:58:20.740 INFO:teuthology.orchestra.run.smithi146.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T10:58:20.743 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T10:58:20.766 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T10:58:20.863 INFO:teuthology.orchestra.run.smithi146.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T10:58:20.866 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T10:58:20.889 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T10:58:20.890 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T10:58:20.938 INFO:teuthology.orchestra.run.smithi069.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T10:58:20.950 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-25T10:58:20.997 INFO:teuthology.orchestra.run.smithi069.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-25T10:58:21.010 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T10:58:21.059 INFO:teuthology.orchestra.run.smithi146.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T10:58:21.071 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-25T10:58:21.118 INFO:teuthology.orchestra.run.smithi146.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-25T10:58:21.131 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T10:58:21.131 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi069.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi146.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T10:58:21.131 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T10:58:21.154 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:58:21.155 INFO:tasks.ceph:role: osd.0 2024-04-25T10:58:21.155 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi069.front.sepia.ceph.com 2024-04-25T10:58:21.155 DEBUG:teuthology.orchestra.run.smithi069:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T10:58:21.225 INFO:teuthology.orchestra.run.smithi069.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:58:21.225 INFO:teuthology.orchestra.run.smithi069.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:58:21.225 INFO:teuthology.orchestra.run.smithi069.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:58:21.225 INFO:teuthology.orchestra.run.smithi069.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:58:21.225 INFO:teuthology.orchestra.run.smithi069.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:58:21.225 INFO:teuthology.orchestra.run.smithi069.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:58:21.225 INFO:teuthology.orchestra.run.smithi069.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:58:21.225 INFO:teuthology.orchestra.run.smithi069.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:58:21.225 INFO:teuthology.orchestra.run.smithi069.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:58:21.225 INFO:teuthology.orchestra.run.smithi069.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:58:21.724 INFO:teuthology.orchestra.run.smithi069.stdout:Discarding blocks...Done. 2024-04-25T10:58:21.725 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi069.front.sepia.ceph.com -o noatime 2024-04-25T10:58:21.726 DEBUG:teuthology.orchestra.run.smithi069:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T10:58:21.828 DEBUG:teuthology.orchestra.run.smithi069:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T10:58:21.859 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T10:58:21.924 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:58:21.924 INFO:tasks.ceph:role: osd.1 2024-04-25T10:58:21.924 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi069.front.sepia.ceph.com 2024-04-25T10:58:21.924 DEBUG:teuthology.orchestra.run.smithi069:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T10:58:21.996 INFO:teuthology.orchestra.run.smithi069.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:58:21.997 INFO:teuthology.orchestra.run.smithi069.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:58:21.997 INFO:teuthology.orchestra.run.smithi069.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:58:21.997 INFO:teuthology.orchestra.run.smithi069.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:58:21.997 INFO:teuthology.orchestra.run.smithi069.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:58:21.997 INFO:teuthology.orchestra.run.smithi069.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:58:21.997 INFO:teuthology.orchestra.run.smithi069.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:58:21.997 INFO:teuthology.orchestra.run.smithi069.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:58:21.997 INFO:teuthology.orchestra.run.smithi069.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:58:21.997 INFO:teuthology.orchestra.run.smithi069.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:58:22.441 INFO:teuthology.orchestra.run.smithi069.stdout:Discarding blocks...Done. 2024-04-25T10:58:22.443 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi069.front.sepia.ceph.com -o noatime 2024-04-25T10:58:22.443 DEBUG:teuthology.orchestra.run.smithi069:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T10:58:22.515 DEBUG:teuthology.orchestra.run.smithi069:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T10:58:22.583 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T10:58:22.647 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:58:22.647 INFO:tasks.ceph:role: osd.2 2024-04-25T10:58:22.648 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi069.front.sepia.ceph.com 2024-04-25T10:58:22.648 DEBUG:teuthology.orchestra.run.smithi069:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T10:58:22.718 INFO:teuthology.orchestra.run.smithi069.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:58:22.718 INFO:teuthology.orchestra.run.smithi069.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:58:22.718 INFO:teuthology.orchestra.run.smithi069.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:58:22.718 INFO:teuthology.orchestra.run.smithi069.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:58:22.719 INFO:teuthology.orchestra.run.smithi069.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:58:22.719 INFO:teuthology.orchestra.run.smithi069.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:58:22.719 INFO:teuthology.orchestra.run.smithi069.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:58:22.719 INFO:teuthology.orchestra.run.smithi069.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:58:22.719 INFO:teuthology.orchestra.run.smithi069.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:58:22.719 INFO:teuthology.orchestra.run.smithi069.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:58:23.198 INFO:teuthology.orchestra.run.smithi069.stdout:Discarding blocks...Done. 2024-04-25T10:58:23.199 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi069.front.sepia.ceph.com -o noatime 2024-04-25T10:58:23.200 DEBUG:teuthology.orchestra.run.smithi069:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T10:58:23.303 DEBUG:teuthology.orchestra.run.smithi069:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T10:58:23.333 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T10:58:23.397 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:58:23.397 INFO:tasks.ceph:role: osd.3 2024-04-25T10:58:23.397 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi069.front.sepia.ceph.com 2024-04-25T10:58:23.397 DEBUG:teuthology.orchestra.run.smithi069:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T10:58:23.470 INFO:teuthology.orchestra.run.smithi069.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:58:23.470 INFO:teuthology.orchestra.run.smithi069.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:58:23.470 INFO:teuthology.orchestra.run.smithi069.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:58:23.471 INFO:teuthology.orchestra.run.smithi069.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:58:23.471 INFO:teuthology.orchestra.run.smithi069.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:58:23.471 INFO:teuthology.orchestra.run.smithi069.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:58:23.471 INFO:teuthology.orchestra.run.smithi069.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:58:23.471 INFO:teuthology.orchestra.run.smithi069.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:58:23.471 INFO:teuthology.orchestra.run.smithi069.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:58:23.471 INFO:teuthology.orchestra.run.smithi069.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:58:23.998 INFO:teuthology.orchestra.run.smithi069.stdout:Discarding blocks...Done. 2024-04-25T10:58:24.001 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi069.front.sepia.ceph.com -o noatime 2024-04-25T10:58:24.001 DEBUG:teuthology.orchestra.run.smithi069:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T10:58:24.147 DEBUG:teuthology.orchestra.run.smithi069:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T10:58:24.178 DEBUG:teuthology.orchestra.run.smithi069:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:58:24.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:24.286+0000 7fb80dfb1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T10:58:24.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:24.291+0000 7fb80dfb1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T10:58:24.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:24.292+0000 7fb80dfb1640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:58:24.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:24.292+0000 7fb80dfb1640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:58:24.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:24.292+0000 7fb80dfb1640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:58:24.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:24.299+0000 7fb80dfb1640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T10:58:27.148 DEBUG:teuthology.orchestra.run.smithi069:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T10:58:27.175 DEBUG:teuthology.orchestra.run.smithi069:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:58:27.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:27.273+0000 7fa95bfd6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T10:58:27.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:27.274+0000 7fa95bfd6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T10:58:27.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:27.274+0000 7fa95bfd6640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:58:27.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:27.274+0000 7fa95bfd6640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:58:27.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:27.274+0000 7fa95bfd6640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:58:27.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:27.274+0000 7fa95bfd6640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T10:58:30.120 DEBUG:teuthology.orchestra.run.smithi069:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T10:58:30.145 DEBUG:teuthology.orchestra.run.smithi069:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:58:30.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:30.239+0000 7f5653933640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T10:58:30.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:30.239+0000 7f5653933640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T10:58:30.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:30.239+0000 7f5653933640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:58:30.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:30.239+0000 7f5653933640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:58:30.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:30.239+0000 7f5653933640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:58:30.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:30.240+0000 7f5653933640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T10:58:33.095 DEBUG:teuthology.orchestra.run.smithi069:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T10:58:33.122 DEBUG:teuthology.orchestra.run.smithi069:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:58:33.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:33.220+0000 7f01a4411640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T10:58:33.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:33.220+0000 7f01a4411640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T10:58:33.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:33.220+0000 7f01a4411640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:58:33.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:33.220+0000 7f01a4411640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:58:33.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:33.220+0000 7f01a4411640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:58:33.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:33.221+0000 7f01a4411640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T10:58:36.074 DEBUG:teuthology.orchestra.run.smithi069:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T10:58:36.101 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T10:58:36.131 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:58:36.131 INFO:tasks.ceph:role: osd.4 2024-04-25T10:58:36.131 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi146.front.sepia.ceph.com 2024-04-25T10:58:36.131 DEBUG:teuthology.orchestra.run.smithi146:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T10:58:36.206 INFO:teuthology.orchestra.run.smithi146.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:58:36.206 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:58:36.206 INFO:teuthology.orchestra.run.smithi146.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:58:36.206 INFO:teuthology.orchestra.run.smithi146.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:58:36.207 INFO:teuthology.orchestra.run.smithi146.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:58:36.207 INFO:teuthology.orchestra.run.smithi146.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:58:36.207 INFO:teuthology.orchestra.run.smithi146.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:58:36.207 INFO:teuthology.orchestra.run.smithi146.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:58:36.207 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:58:36.207 INFO:teuthology.orchestra.run.smithi146.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:58:36.705 INFO:teuthology.orchestra.run.smithi146.stdout:Discarding blocks...Done. 2024-04-25T10:58:36.706 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi146.front.sepia.ceph.com -o noatime 2024-04-25T10:58:36.706 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T10:58:36.811 DEBUG:teuthology.orchestra.run.smithi146:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T10:58:36.843 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T10:58:36.908 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:58:36.908 INFO:tasks.ceph:role: osd.5 2024-04-25T10:58:36.908 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi146.front.sepia.ceph.com 2024-04-25T10:58:36.908 DEBUG:teuthology.orchestra.run.smithi146:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T10:58:36.980 INFO:teuthology.orchestra.run.smithi146.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:58:36.981 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:58:36.981 INFO:teuthology.orchestra.run.smithi146.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:58:36.981 INFO:teuthology.orchestra.run.smithi146.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:58:36.981 INFO:teuthology.orchestra.run.smithi146.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:58:36.981 INFO:teuthology.orchestra.run.smithi146.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:58:36.981 INFO:teuthology.orchestra.run.smithi146.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:58:36.981 INFO:teuthology.orchestra.run.smithi146.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:58:36.981 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:58:36.981 INFO:teuthology.orchestra.run.smithi146.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:58:37.445 INFO:teuthology.orchestra.run.smithi146.stdout:Discarding blocks...Done. 2024-04-25T10:58:37.447 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi146.front.sepia.ceph.com -o noatime 2024-04-25T10:58:37.447 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T10:58:37.520 DEBUG:teuthology.orchestra.run.smithi146:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T10:58:37.588 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T10:58:37.651 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:58:37.652 INFO:tasks.ceph:role: osd.6 2024-04-25T10:58:37.652 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi146.front.sepia.ceph.com 2024-04-25T10:58:37.652 DEBUG:teuthology.orchestra.run.smithi146:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T10:58:37.723 INFO:teuthology.orchestra.run.smithi146.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:58:37.723 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:58:37.723 INFO:teuthology.orchestra.run.smithi146.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:58:37.723 INFO:teuthology.orchestra.run.smithi146.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:58:37.723 INFO:teuthology.orchestra.run.smithi146.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:58:37.723 INFO:teuthology.orchestra.run.smithi146.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:58:37.723 INFO:teuthology.orchestra.run.smithi146.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:58:37.724 INFO:teuthology.orchestra.run.smithi146.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:58:37.724 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:58:37.724 INFO:teuthology.orchestra.run.smithi146.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:58:38.208 INFO:teuthology.orchestra.run.smithi146.stdout:Discarding blocks...Done. 2024-04-25T10:58:38.211 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi146.front.sepia.ceph.com -o noatime 2024-04-25T10:58:38.211 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T10:58:38.311 DEBUG:teuthology.orchestra.run.smithi146:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T10:58:38.341 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T10:58:38.405 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:58:38.405 INFO:tasks.ceph:role: osd.7 2024-04-25T10:58:38.405 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi146.front.sepia.ceph.com 2024-04-25T10:58:38.405 DEBUG:teuthology.orchestra.run.smithi146:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T10:58:38.476 INFO:teuthology.orchestra.run.smithi146.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:58:38.476 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:58:38.476 INFO:teuthology.orchestra.run.smithi146.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:58:38.476 INFO:teuthology.orchestra.run.smithi146.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:58:38.476 INFO:teuthology.orchestra.run.smithi146.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:58:38.476 INFO:teuthology.orchestra.run.smithi146.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:58:38.476 INFO:teuthology.orchestra.run.smithi146.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:58:38.476 INFO:teuthology.orchestra.run.smithi146.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:58:38.476 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:58:38.476 INFO:teuthology.orchestra.run.smithi146.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:58:38.983 INFO:teuthology.orchestra.run.smithi146.stdout:Discarding blocks...Done. 2024-04-25T10:58:38.984 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi146.front.sepia.ceph.com -o noatime 2024-04-25T10:58:38.985 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T10:58:39.129 DEBUG:teuthology.orchestra.run.smithi146:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T10:58:39.161 DEBUG:teuthology.orchestra.run.smithi146:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:58:39.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:39.261+0000 7fc747c30640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T10:58:39.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:39.268+0000 7fc747c30640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T10:58:39.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:39.268+0000 7fc747c30640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:58:39.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:39.268+0000 7fc747c30640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:58:39.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:39.268+0000 7fc747c30640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:58:39.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:39.268+0000 7fc747c30640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T10:58:42.118 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T10:58:42.146 DEBUG:teuthology.orchestra.run.smithi146:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:58:42.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:42.246+0000 7f67d0043640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T10:58:42.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:42.246+0000 7f67d0043640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T10:58:42.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:42.246+0000 7f67d0043640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:58:42.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:42.246+0000 7f67d0043640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:58:42.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:42.246+0000 7f67d0043640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:58:42.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:42.247+0000 7f67d0043640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T10:58:45.109 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T10:58:45.136 DEBUG:teuthology.orchestra.run.smithi146:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:58:45.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:45.235+0000 7fc18fcf0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T10:58:45.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:45.235+0000 7fc18fcf0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T10:58:45.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:45.235+0000 7fc18fcf0640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:58:45.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:45.235+0000 7fc18fcf0640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:58:45.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:45.235+0000 7fc18fcf0640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:58:45.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:45.236+0000 7fc18fcf0640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T10:58:47.994 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:58:47.997 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:58:48.111 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T10:58:48.177 DEBUG:teuthology.orchestra.run.smithi146:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:58:48.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:48.274+0000 7f9723413640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T10:58:48.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:48.274+0000 7f9723413640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T10:58:48.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:48.274+0000 7f9723413640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:58:48.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:48.274+0000 7f9723413640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:58:48.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:48.274+0000 7f9723413640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:58:48.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:48.275+0000 7f9723413640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T10:58:51.134 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T10:58:51.162 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T10:58:51.163 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:58:51.163 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T10:58:51.191 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:58:51.192 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T10:58:51.259 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:58:51.259 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T10:58:51.325 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:58:51.325 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T10:58:51.390 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:58:51.391 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T10:58:51.455 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:58:51.455 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T10:58:51.519 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:58:51.520 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T10:58:51.583 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T10:58:51.583 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T10:58:51.612 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T10:58:51.612 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T10:58:51.680 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T10:58:51.680 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T10:58:51.748 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T10:58:51.748 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T10:58:51.815 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T10:58:51.815 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T10:58:51.881 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T10:58:51.881 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T10:58:51.946 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T10:58:51.946 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T10:58:52.011 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:58:52.011 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T10:58:52.027 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:58:52.027 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-25T10:58:52.083 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T10:58:52.083 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T10:58:52.099 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T10:58:52.099 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-25T10:58:52.154 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T10:58:52.155 DEBUG:teuthology.orchestra.run.smithi069:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T10:58:52.157 DEBUG:teuthology.orchestra.run.smithi146:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T10:58:52.198 INFO:teuthology.orchestra.run.smithi069.stdout:[mgr.y] 2024-04-25T10:58:52.198 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQBMNypm4Gx8FxAAsS/vgwzV6B3i6oVbp+gRqQ== 2024-04-25T10:58:52.199 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.a] 2024-04-25T10:58:52.199 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQBMNypmnESqHxAAb0BNMP4l0t7vd3uAlQOxig== 2024-04-25T10:58:52.199 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.c] 2024-04-25T10:58:52.199 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQBMNypmtOQPJxAAkoL9qh4tJqPgMUnKIkCoMQ== 2024-04-25T10:58:52.199 INFO:teuthology.orchestra.run.smithi069.stdout:[osd.0] 2024-04-25T10:58:52.199 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQBQNypmfvAiERAAoPcVgX0yuZPAEj6HhgxJ1g== 2024-04-25T10:58:52.199 INFO:teuthology.orchestra.run.smithi069.stdout:[osd.1] 2024-04-25T10:58:52.199 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQBTNypm1C9kEBAACwOq6Mmuxiw2Jw1tax/Sbw== 2024-04-25T10:58:52.199 INFO:teuthology.orchestra.run.smithi069.stdout:[osd.2] 2024-04-25T10:58:52.199 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQBWNypm32NVDhAAdRFsWgkXHNpy9Cpl+ntR1w== 2024-04-25T10:58:52.199 INFO:teuthology.orchestra.run.smithi069.stdout:[osd.3] 2024-04-25T10:58:52.199 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQBZNypm7IIzDRAAfXeNYJ42bXE4TX2F8EAevA== 2024-04-25T10:58:52.199 INFO:teuthology.orchestra.run.smithi069.stdout:[mgr.x] 2024-04-25T10:58:52.199 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQBMNypmq1MRHBAACDY/Y25XF2oiwMi3/qR7wQ== 2024-04-25T10:58:52.200 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.b] 2024-04-25T10:58:52.200 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQBMNypmRLQkLBAAzJk7Zak+aQP4Xi6YayKdUw== 2024-04-25T10:58:52.200 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.d] 2024-04-25T10:58:52.200 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQBMNypmI/6FMxAAyVFxJEwF2XnZ1Nb7MeIYGQ== 2024-04-25T10:58:52.200 INFO:teuthology.orchestra.run.smithi069.stdout:[osd.4] 2024-04-25T10:58:52.200 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQBfNypmFTGrDxAAITl0VOssjLlcOYiug6B0pA== 2024-04-25T10:58:52.200 INFO:teuthology.orchestra.run.smithi069.stdout:[osd.5] 2024-04-25T10:58:52.200 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQBiNypmRgHLDhAAaCiszWXZ8DDTHuapyFE3lQ== 2024-04-25T10:58:52.200 INFO:teuthology.orchestra.run.smithi069.stdout:[osd.6] 2024-04-25T10:58:52.200 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQBlNypme8AZDhAA1syulvLhqH27AbYYuSWnVw== 2024-04-25T10:58:52.200 INFO:teuthology.orchestra.run.smithi069.stdout:[osd.7] 2024-04-25T10:58:52.200 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQBoNypmtrloEBAAm8UktQI0CNss2Goap/EddA== 2024-04-25T10:58:52.200 INFO:teuthology.orchestra.run.smithi069.stdout:[client.0] 2024-04-25T10:58:52.200 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQBMNypmtv/zNxAAz2JcqUNyHu2ipkrbl8ONng== 2024-04-25T10:58:52.200 INFO:teuthology.orchestra.run.smithi069.stdout:[client.2] 2024-04-25T10:58:52.201 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQBMNypmTQeEOxAAqM/YzZlM2cz7zo2g78NCMw== 2024-04-25T10:58:52.201 INFO:teuthology.orchestra.run.smithi069.stdout:[client.1] 2024-04-25T10:58:52.201 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQBNNypm5U2DAxAAG+NYDPdMtwS4CvAtln+8Aw== 2024-04-25T10:58:52.201 INFO:teuthology.orchestra.run.smithi069.stdout:[client.3] 2024-04-25T10:58:52.201 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQBNNypm7oIZBxAAABnro3onV9BFztCKMJapkA== 2024-04-25T10:58:52.217 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr.y] 2024-04-25T10:58:52.218 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQBMNypm4Gx8FxAAsS/vgwzV6B3i6oVbp+gRqQ== 2024-04-25T10:58:52.218 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.a] 2024-04-25T10:58:52.218 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQBMNypmnESqHxAAb0BNMP4l0t7vd3uAlQOxig== 2024-04-25T10:58:52.218 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.c] 2024-04-25T10:58:52.218 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQBMNypmtOQPJxAAkoL9qh4tJqPgMUnKIkCoMQ== 2024-04-25T10:58:52.218 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.0] 2024-04-25T10:58:52.218 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQBQNypmfvAiERAAoPcVgX0yuZPAEj6HhgxJ1g== 2024-04-25T10:58:52.218 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.1] 2024-04-25T10:58:52.218 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQBTNypm1C9kEBAACwOq6Mmuxiw2Jw1tax/Sbw== 2024-04-25T10:58:52.218 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.2] 2024-04-25T10:58:52.218 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQBWNypm32NVDhAAdRFsWgkXHNpy9Cpl+ntR1w== 2024-04-25T10:58:52.218 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.3] 2024-04-25T10:58:52.218 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQBZNypm7IIzDRAAfXeNYJ42bXE4TX2F8EAevA== 2024-04-25T10:58:52.219 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr.x] 2024-04-25T10:58:52.219 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQBMNypmq1MRHBAACDY/Y25XF2oiwMi3/qR7wQ== 2024-04-25T10:58:52.219 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.b] 2024-04-25T10:58:52.219 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQBMNypmRLQkLBAAzJk7Zak+aQP4Xi6YayKdUw== 2024-04-25T10:58:52.219 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.d] 2024-04-25T10:58:52.219 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQBMNypmI/6FMxAAyVFxJEwF2XnZ1Nb7MeIYGQ== 2024-04-25T10:58:52.219 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.4] 2024-04-25T10:58:52.219 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQBfNypmFTGrDxAAITl0VOssjLlcOYiug6B0pA== 2024-04-25T10:58:52.219 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.5] 2024-04-25T10:58:52.219 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQBiNypmRgHLDhAAaCiszWXZ8DDTHuapyFE3lQ== 2024-04-25T10:58:52.219 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.6] 2024-04-25T10:58:52.219 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQBlNypme8AZDhAA1syulvLhqH27AbYYuSWnVw== 2024-04-25T10:58:52.219 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.7] 2024-04-25T10:58:52.220 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQBoNypmtrloEBAAm8UktQI0CNss2Goap/EddA== 2024-04-25T10:58:52.220 INFO:teuthology.orchestra.run.smithi146.stdout:[client.0] 2024-04-25T10:58:52.220 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQBMNypmtv/zNxAAz2JcqUNyHu2ipkrbl8ONng== 2024-04-25T10:58:52.220 INFO:teuthology.orchestra.run.smithi146.stdout:[client.2] 2024-04-25T10:58:52.220 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQBMNypmTQeEOxAAqM/YzZlM2cz7zo2g78NCMw== 2024-04-25T10:58:52.220 INFO:teuthology.orchestra.run.smithi146.stdout:[client.1] 2024-04-25T10:58:52.220 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQBNNypm5U2DAxAAG+NYDPdMtwS4CvAtln+8Aw== 2024-04-25T10:58:52.220 INFO:teuthology.orchestra.run.smithi146.stdout:[client.3] 2024-04-25T10:58:52.220 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQBNNypm7oIZBxAAABnro3onV9BFztCKMJapkA== 2024-04-25T10:58:52.221 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:58:52.245 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:58:52.322 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:58:52.363 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:58:52.424 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:58:52.426 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:58:52.487 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:58:52.489 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:58:52.548 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:58:52.550 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:58:52.608 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:58:52.611 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:58:52.670 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:58:52.673 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:58:52.731 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:58:52.733 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:58:52.794 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:58:52.796 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:58:52.858 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:58:52.860 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:58:52.923 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:58:52.925 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:58:52.984 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:58:52.986 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:58:53.045 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:58:53.048 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:58:53.106 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:58:53.108 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:58:53.167 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:58:53.169 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:58:53.231 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:58:53.233 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:58:53.294 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:58:53.297 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:58:53.359 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:58:53.361 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:58:53.423 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T10:58:53.423 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T10:58:53.446 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:58:53.550 DEBUG:teuthology.orchestra.run.smithi069:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T10:58:53.573 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T10:58:53.596 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:58:53.702 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T10:58:53.726 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T10:58:53.792 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:58:53.895 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T10:58:53.918 DEBUG:teuthology.orchestra.run.smithi069:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:58:53.921 DEBUG:teuthology.orchestra.run.smithi146:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:58:53.972 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T10:58:53.973 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T10:58:53.973 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T10:58:53.978 INFO:tasks.ceph.mon.a:Started 2024-04-25T10:58:53.978 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T10:58:53.978 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T10:58:54.016 INFO:tasks.ceph.mon.b:Started 2024-04-25T10:58:54.016 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T10:58:54.017 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T10:58:54.019 INFO:tasks.ceph.mon.c:Started 2024-04-25T10:58:54.019 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T10:58:54.020 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T10:58:54.020 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T10:58:54.022 INFO:tasks.ceph.mgr.y:Started 2024-04-25T10:58:54.023 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T10:58:54.023 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T10:58:54.025 INFO:tasks.ceph.mgr.x:Started 2024-04-25T10:58:54.025 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T10:58:54.025 DEBUG:teuthology.orchestra.run.smithi069:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T10:58:54.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.135+0000 7f3e74e51640 1 Processor -- start 2024-04-25T10:58:54.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.135+0000 7f3e74e51640 1 -- start start 2024-04-25T10:58:54.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.137+0000 7f3e74e51640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f3e7006ccd0 0x7f3e7015e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:54.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.137+0000 7f3e74e51640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e70163fa0 0x7f3e701643b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:54.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.137+0000 7f3e74e51640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e701609b0 con 0x7f3e70161710 2024-04-25T10:58:54.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.137+0000 7f3e74e51640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f3e70164cd0 con 0x7f3e7006ccd0 2024-04-25T10:58:54.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.137+0000 7f3e74e51640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3e70160c30 con 0x7f3e70163fa0 2024-04-25T10:58:54.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.137+0000 7f3e6e59c640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f3e70161710 0x7f3e7006c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:43732/0 (socket says 172.21.15.69:43732) 2024-04-25T10:58:54.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.137+0000 7f3e6dd9b640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f3e7006ccd0 0x7f3e7015e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:54.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.137+0000 7f3e6e59c640 1 -- 172.21.15.69:0/2254510184 learned_addr learned my addr 172.21.15.69:0/2254510184 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:58:54.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.137+0000 7f3e6dd9b640 1 -- 172.21.15.69:0/2254510184 >> v2:172.21.15.69:3300/0 conn(0x7f3e7006ccd0 msgr2=0x7f3e7015e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T10:58:54.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.137+0000 7f3e6dd9b640 1 -- 172.21.15.69:0/2254510184 >> v2:172.21.15.69:3300/0 conn(0x7f3e7006ccd0 msgr2=0x7f3e7015e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:58:54.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.137+0000 7f3e6dd9b640 1 --2- 172.21.15.69:0/2254510184 >> v2:172.21.15.69:3300/0 conn(0x7f3e7006ccd0 0x7f3e7015e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:58:54.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.137+0000 7f3e6dd9b640 1 --2- 172.21.15.69:0/2254510184 >> v2:172.21.15.69:3300/0 conn(0x7f3e7006ccd0 0x7f3e7015e080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:58:54.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.137+0000 7f3e6ed9d640 1 --2- 172.21.15.69:0/2254510184 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e70163fa0 0x7f3e701643b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:54.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.138+0000 7f3e6ed9d640 1 -- 172.21.15.69:0/2254510184 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e70163fa0 msgr2=0x7f3e701643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:58:54.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.138+0000 7f3e6ed9d640 1 -- 172.21.15.69:0/2254510184 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e70163fa0 msgr2=0x7f3e701643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:58:54.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.138+0000 7f3e6ed9d640 1 --2- 172.21.15.69:0/2254510184 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e70163fa0 0x7f3e701643b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:58:54.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.138+0000 7f3e6ed9d640 1 --2- 172.21.15.69:0/2254510184 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e70163fa0 0x7f3e701643b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:58:54.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.303+0000 7f3e6d59a640 1 -- 172.21.15.69:0/2254510184 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2761455142 0 0) 0x7f3e60001490 con 0x7f3e70161710 2024-04-25T10:58:54.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.303+0000 7f3e6d59a640 1 -- 172.21.15.69:0/2254510184 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e58003520 con 0x7f3e70161710 2024-04-25T10:58:54.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.304+0000 7f3e6d59a640 1 -- 172.21.15.69:0/2254510184 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1778178170 0 0) 0x7f3e58003520 con 0x7f3e70161710 2024-04-25T10:58:54.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.304+0000 7f3e6d59a640 1 -- 172.21.15.69:0/2254510184 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e70163fa0 msgr2=0x7f3e701643b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:58:54.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.304+0000 7f3e6d59a640 1 --2- 172.21.15.69:0/2254510184 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e70163fa0 0x7f3e701643b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:54.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.304+0000 7f3e6d59a640 1 -- 172.21.15.69:0/2254510184 >> v2:172.21.15.69:3300/0 conn(0x7f3e7006ccd0 msgr2=0x7f3e7015e080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:58:54.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.304+0000 7f3e6d59a640 1 --2- 172.21.15.69:0/2254510184 >> v2:172.21.15.69:3300/0 conn(0x7f3e7006ccd0 0x7f3e7015e080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:54.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.304+0000 7f3e6d59a640 1 -- 172.21.15.69:0/2254510184 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e70160f10 con 0x7f3e70161710 2024-04-25T10:58:54.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.304+0000 7f3e6d59a640 1 -- 172.21.15.69:0/2254510184 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7f3e60001bc0 con 0x7f3e70161710 2024-04-25T10:58:54.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.304+0000 7f3e6d59a640 1 -- 172.21.15.69:0/2254510184 >> v1:172.21.15.69:6789/0 conn(0x7f3e70161710 legacy=0x7f3e7006c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:54.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.304+0000 7f3e6d59a640 1 --2- 172.21.15.69:0/2254510184 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e70163fa0 0x7f3e580039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:54.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.304+0000 7f3e6d59a640 1 --2- 172.21.15.69:0/2254510184 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e58003fe0 0x7f3e58004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:54.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.304+0000 7f3e6d59a640 1 --2- 172.21.15.69:0/2254510184 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e58004d40 0x7f3e58005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:54.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.304+0000 7f3e6d59a640 1 -- 172.21.15.69:0/2254510184 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3e70164cd0 con 0x7f3e58003fe0 2024-04-25T10:58:54.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.304+0000 7f3e6d59a640 1 -- 172.21.15.69:0/2254510184 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3e70160c30 con 0x7f3e58004d40 2024-04-25T10:58:54.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.304+0000 7f3e6d59a640 1 -- 172.21.15.69:0/2254510184 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3e5800bc30 con 0x7f3e70163fa0 2024-04-25T10:58:54.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.304+0000 7f3e6e59c640 1 --2- 172.21.15.69:0/2254510184 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e70163fa0 0x7f3e580039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:54.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.304+0000 7f3e6ed9d640 1 --2- 172.21.15.69:0/2254510184 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e58004d40 0x7f3e58005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:54.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.304+0000 7f3e6dd9b640 1 --2- 172.21.15.69:0/2254510184 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e58003fe0 0x7f3e58004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:54.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.305+0000 7f3e6ed9d640 1 -- 172.21.15.69:0/2254510184 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e70163fa0 msgr2=0x7f3e580039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:54.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.305+0000 7f3e6ed9d640 1 --2- 172.21.15.69:0/2254510184 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e70163fa0 0x7f3e580039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:54.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.305+0000 7f3e6ed9d640 1 -- 172.21.15.69:0/2254510184 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e58003fe0 msgr2=0x7f3e58004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:54.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.305+0000 7f3e6ed9d640 1 --2- 172.21.15.69:0/2254510184 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e58003fe0 0x7f3e58004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:54.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.305+0000 7f3e6ed9d640 1 -- 172.21.15.69:0/2254510184 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e58003520 con 0x7f3e58004d40 2024-04-25T10:58:54.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.305+0000 7f3e6dd9b640 1 --2- 172.21.15.69:0/2254510184 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e58003fe0 0x7f3e58004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:58:54.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.305+0000 7f3e6ed9d640 1 --2- 172.21.15.69:0/2254510184 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e58004d40 0x7f3e58005150 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3e700d3ff0 tx=0x7f3e58005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb36361ddb5fcbe5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:54.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.306+0000 7f3e6d59a640 1 -- 172.21.15.69:0/2254510184 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e68030070 con 0x7f3e58004d40 2024-04-25T10:58:54.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.306+0000 7f3e6d59a640 1 -- 172.21.15.69:0/2254510184 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e6802ad40 con 0x7f3e58004d40 2024-04-25T10:58:54.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.306+0000 7f3e6d59a640 1 -- 172.21.15.69:0/2254510184 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6800b5e0 con 0x7f3e58004d40 2024-04-25T10:58:54.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.307+0000 7f3e74e51640 1 -- 172.21.15.69:0/2254510184 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e58004d40 msgr2=0x7f3e58005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:54.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.307+0000 7f3e74e51640 1 --2- 172.21.15.69:0/2254510184 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e58004d40 0x7f3e58005150 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3e700d3ff0 tx=0x7f3e58005830 comp rx=0 tx=0).stop 2024-04-25T10:58:54.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.307+0000 7f3e6ed9d640 1 -- 172.21.15.69:0/2254510184 reap_dead start 2024-04-25T10:58:54.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.307+0000 7f3e74e51640 1 -- 172.21.15.69:0/2254510184 shutdown_connections 2024-04-25T10:58:54.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.307+0000 7f3e74e51640 1 -- 172.21.15.69:0/2254510184 >> 172.21.15.69:0/2254510184 conn(0x7f3e700d5d90 msgr2=0x7f3e700d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:54.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.307+0000 7f3e74e51640 1 -- 172.21.15.69:0/2254510184 shutdown_connections 2024-04-25T10:58:54.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.307+0000 7f3e74e51640 1 -- 172.21.15.69:0/2254510184 wait complete. 2024-04-25T10:58:54.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.309+0000 7f3e74e51640 1 Processor -- start 2024-04-25T10:58:54.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.309+0000 7f3e74e51640 1 -- start start 2024-04-25T10:58:54.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.311+0000 7f3e74e51640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e58004d40 0x7f3e700c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:54.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.311+0000 7f3e74e51640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e700ce6c0 0x7f3e700cead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:54.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.311+0000 7f3e74e51640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e700cf010 0x7f3e701f5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:54.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.311+0000 7f3e74e51640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3e70164530 con 0x7f3e700cf010 2024-04-25T10:58:54.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.311+0000 7f3e74e51640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3e701643b0 con 0x7f3e700ce6c0 2024-04-25T10:58:54.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.311+0000 7f3e74e51640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3e701646b0 con 0x7f3e58004d40 2024-04-25T10:58:54.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.311+0000 7f3e6ed9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e700cf010 0x7f3e701f5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:54.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.311+0000 7f3e6e59c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e58004d40 0x7f3e700c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:54.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.311+0000 7f3e6ed9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e700cf010 0x7f3e701f5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42870/0 (socket says 172.21.15.69:42870) 2024-04-25T10:58:54.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.311+0000 7f3e6ed9d640 1 -- 172.21.15.69:0/847889110 learned_addr learned my addr 172.21.15.69:0/847889110 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:58:54.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.311+0000 7f3e6dd9b640 1 --2- 172.21.15.69:0/847889110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e700ce6c0 0x7f3e700cead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:54.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.311+0000 7f3e6ed9d640 1 -- 172.21.15.69:0/847889110 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e58004d40 msgr2=0x7f3e700c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:54.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.311+0000 7f3e6ed9d640 1 --2- 172.21.15.69:0/847889110 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e58004d40 0x7f3e700c8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:54.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.311+0000 7f3e6ed9d640 1 -- 172.21.15.69:0/847889110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e700ce6c0 msgr2=0x7f3e700cead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:54.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.311+0000 7f3e6ed9d640 1 --2- 172.21.15.69:0/847889110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e700ce6c0 0x7f3e700cead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:54.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.311+0000 7f3e6ed9d640 1 -- 172.21.15.69:0/847889110 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e7015c450 con 0x7f3e700cf010 2024-04-25T10:58:54.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.311+0000 7f3e6ed9d640 1 --2- 172.21.15.69:0/847889110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e700cf010 0x7f3e701f5f80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3e6800baa0 tx=0x7f3e68004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:54.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.311+0000 7f3e5effd640 1 -- 172.21.15.69:0/847889110 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6400b5e0 con 0x7f3e700cf010 2024-04-25T10:58:54.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.311+0000 7f3e5effd640 1 -- 172.21.15.69:0/847889110 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e68045070 con 0x7f3e700cf010 2024-04-25T10:58:54.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.312+0000 7f3e74e51640 1 -- 172.21.15.69:0/847889110 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e7015be90 con 0x7f3e700cf010 2024-04-25T10:58:54.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.312+0000 7f3e5effd640 1 -- 172.21.15.69:0/847889110 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6800b5e0 con 0x7f3e700cf010 2024-04-25T10:58:54.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.312+0000 7f3e74e51640 1 -- 172.21.15.69:0/847889110 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e700cdec0 con 0x7f3e700cf010 2024-04-25T10:58:54.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.312+0000 7f3e5effd640 1 -- 172.21.15.69:0/847889110 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3e6803f6b0 con 0x7f3e700cf010 2024-04-25T10:58:54.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.312+0000 7f3e5effd640 1 -- 172.21.15.69:0/847889110 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3e6804a8c0 con 0x7f3e700cf010 2024-04-25T10:58:54.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.312+0000 7f3e74e51640 1 -- 172.21.15.69:0/847889110 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e7014aa00 con 0x7f3e700cf010 2024-04-25T10:58:54.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.315+0000 7f3e5effd640 1 -- 172.21.15.69:0/847889110 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f3e6804a460 con 0x7f3e700cf010 2024-04-25T10:58:54.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.381+0000 7f3e74e51640 1 -- 172.21.15.69:0/847889110 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f3e70050740 con 0x7f3e700cf010 2024-04-25T10:58:54.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.382+0000 7f3e5effd640 1 -- 172.21.15.69:0/847889110 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f3e6803fb30 con 0x7f3e700cf010 2024-04-25T10:58:54.383 INFO:teuthology.orchestra.run.smithi069.stderr:adjusted tunables profile to default 2024-04-25T10:58:54.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.383+0000 7f3e74e51640 1 -- 172.21.15.69:0/847889110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e700cf010 msgr2=0x7f3e701f5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:54.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.383+0000 7f3e74e51640 1 --2- 172.21.15.69:0/847889110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e700cf010 0x7f3e701f5f80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3e6800baa0 tx=0x7f3e68004730 comp rx=0 tx=0).stop 2024-04-25T10:58:54.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.383+0000 7f3e74e51640 1 -- 172.21.15.69:0/847889110 shutdown_connections 2024-04-25T10:58:54.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.383+0000 7f3e74e51640 1 --2- 172.21.15.69:0/847889110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e700cf010 0x7f3e701f5f80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:54.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.383+0000 7f3e74e51640 1 --2- 172.21.15.69:0/847889110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e700ce6c0 0x7f3e700cead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:54.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.383+0000 7f3e74e51640 1 --2- 172.21.15.69:0/847889110 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e58004d40 0x7f3e700c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:54.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.383+0000 7f3e74e51640 1 -- 172.21.15.69:0/847889110 >> 172.21.15.69:0/847889110 conn(0x7f3e700d5d90 msgr2=0x7f3e700d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:54.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.383+0000 7f3e74e51640 1 -- 172.21.15.69:0/847889110 shutdown_connections 2024-04-25T10:58:54.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:54.383+0000 7f3e74e51640 1 -- 172.21.15.69:0/847889110 wait complete. 2024-04-25T10:58:54.395 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T10:58:54.396 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T10:58:54.396 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:58:54.396 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T10:58:54.412 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T10:58:54.412+0000 7f53b8f19040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:58:54.415 INFO:tasks.ceph.mgr.y.smithi069.stderr:2024-04-25T10:58:54.415+0000 7fecc21ee040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:58:54.420 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:58:54.420 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T10:58:54.483 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:58:54.483 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T10:58:54.546 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:58:54.546 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T10:58:54.559 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T10:58:54.559+0000 7f53b8f19040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:58:54.590 INFO:tasks.ceph.mgr.y.smithi069.stderr:2024-04-25T10:58:54.589+0000 7fecc21ee040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:58:54.611 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T10:58:54.611 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T10:58:54.612 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T10:58:54.612+0000 7f53b8f19040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:58:54.635 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T10:58:54.636 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T10:58:54.647 INFO:tasks.ceph.mgr.y.smithi069.stderr:2024-04-25T10:58:54.646+0000 7fecc21ee040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:58:54.698 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T10:58:54.699 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T10:58:54.763 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T10:58:54.763 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T10:58:54.826 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new 0c380208-30e4-47a1-83b0-423ae5ead112 0 2024-04-25T10:58:54.873 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T10:58:54.873+0000 7f53b8f19040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:58:54.881 INFO:tasks.ceph.mgr.y.smithi069.stderr:2024-04-25T10:58:54.880+0000 7fecc21ee040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:58:54.922 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T10:58:54.922+0000 7f53b8f19040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:58:54.925 INFO:tasks.ceph.mgr.y.smithi069.stderr:2024-04-25T10:58:54.925+0000 7fecc21ee040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:58:54.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.975+0000 7f06e7e34640 1 Processor -- start 2024-04-25T10:58:54.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.976+0000 7f06e7e34640 1 -- start start 2024-04-25T10:58:54.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.977+0000 7f06e7e34640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f06e00b1b20 0x7f06e006ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:54.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.977+0000 7f06e7e34640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f06e00b7e70 0x7f06e00d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:54.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.977+0000 7f06e7e34640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06e00cf960 con 0x7f06e00b3d70 2024-04-25T10:58:54.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.977+0000 7f06e7e34640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f06e006d800 con 0x7f06e00b7e70 2024-04-25T10:58:54.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.977+0000 7f06e7e34640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f06e00b4230 con 0x7f06e00b1b20 2024-04-25T10:58:54.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.977+0000 7f06e5bd0640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f06e00b1b20 0x7f06e006ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:54.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.977+0000 7f06e5bd0640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f06e00b1b20 0x7f06e006ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57300/0 (socket says 172.21.15.146:57300) 2024-04-25T10:58:54.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.977+0000 7f06e5bd0640 1 -- 172.21.15.146:0/2555615578 learned_addr learned my addr 172.21.15.146:0/2555615578 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-04-25T10:58:54.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.977+0000 7f06e63d1640 1 --2- 172.21.15.146:0/2555615578 >> v2:172.21.15.69:3300/0 conn(0x7f06e00b7e70 0x7f06e00d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:54.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.977+0000 7f06e5bd0640 1 -- 172.21.15.146:0/2555615578 >> v2:172.21.15.69:3300/0 conn(0x7f06e00b7e70 msgr2=0x7f06e00d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:54.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.977+0000 7f06e5bd0640 1 --2- 172.21.15.146:0/2555615578 >> v2:172.21.15.69:3300/0 conn(0x7f06e00b7e70 0x7f06e00d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:54.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.977+0000 7f06e5bd0640 1 -- 172.21.15.146:0/2555615578 >> v1:172.21.15.69:6789/0 conn(0x7f06e00b3d70 legacy=0x7f06e00b7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:54.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.977+0000 7f06e5bd0640 1 -- 172.21.15.146:0/2555615578 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e00cf3a0 con 0x7f06e00b1b20 2024-04-25T10:58:54.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.978+0000 7f06e5bd0640 1 --2- 172.21.15.146:0/2555615578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06e00b1b20 0x7f06e006ec50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f06e0073160 tx=0x7f06d002c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c666d7de7a349dd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:54.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.978+0000 7f06e4bce640 1 -- 172.21.15.146:0/2555615578 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0036020 con 0x7f06e00b1b20 2024-04-25T10:58:54.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.978+0000 7f06e4bce640 1 -- 172.21.15.146:0/2555615578 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06d0032640 con 0x7f06e00b1b20 2024-04-25T10:58:54.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.978+0000 7f06e4bce640 1 -- 172.21.15.146:0/2555615578 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0032a10 con 0x7f06e00b1b20 2024-04-25T10:58:54.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.979+0000 7f06e7e34640 1 -- 172.21.15.146:0/2555615578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06e00b1b20 msgr2=0x7f06e006ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:54.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.979+0000 7f06e7e34640 1 --2- 172.21.15.146:0/2555615578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06e00b1b20 0x7f06e006ec50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f06e0073160 tx=0x7f06d002c940 comp rx=0 tx=0).stop 2024-04-25T10:58:54.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.979+0000 7f06e7e34640 1 -- 172.21.15.146:0/2555615578 shutdown_connections 2024-04-25T10:58:54.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.979+0000 7f06e7e34640 1 --2- 172.21.15.146:0/2555615578 >> v2:172.21.15.69:3300/0 conn(0x7f06e00b7e70 0x7f06e00d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:54.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.979+0000 7f06e7e34640 1 --2- 172.21.15.146:0/2555615578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06e00b1b20 0x7f06e006ec50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:54.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.979+0000 7f06e7e34640 1 -- 172.21.15.146:0/2555615578 >> 172.21.15.146:0/2555615578 conn(0x7f06e009b7d0 msgr2=0x7f06e009bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:54.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.979+0000 7f06e7e34640 1 -- 172.21.15.146:0/2555615578 shutdown_connections 2024-04-25T10:58:54.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.979+0000 7f06e7e34640 1 -- 172.21.15.146:0/2555615578 wait complete. 2024-04-25T10:58:54.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.981+0000 7f06e7e34640 1 Processor -- start 2024-04-25T10:58:54.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.981+0000 7f06e7e34640 1 -- start start 2024-04-25T10:58:54.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.982+0000 7f06e7e34640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f06e00b1b20 0x7f06e0151850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:54.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.982+0000 7f06e7e34640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06e0151e10 0x7f06e014f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:54.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.982+0000 7f06e7e34640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06e00c7540 0x7f06e00c7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:54.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.982+0000 7f06e7e34640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f06e015d6a0 con 0x7f06e00c7540 2024-04-25T10:58:54.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.982+0000 7f06e7e34640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f06e015d520 con 0x7f06e0151e10 2024-04-25T10:58:54.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.982+0000 7f06e7e34640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f06e006dc90 con 0x7f06e00b1b20 2024-04-25T10:58:54.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.982+0000 7f06e53cf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06e0151e10 0x7f06e014f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:54.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.982+0000 7f06e5bd0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f06e00b1b20 0x7f06e0151850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:54.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.982+0000 7f06e53cf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06e0151e10 0x7f06e014f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57312/0 (socket says 172.21.15.146:57312) 2024-04-25T10:58:54.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.982+0000 7f06e5bd0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f06e00b1b20 0x7f06e0151850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:43956/0 (socket says 172.21.15.146:43956) 2024-04-25T10:58:54.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.982+0000 7f06e53cf640 1 -- 172.21.15.146:0/3528018339 learned_addr learned my addr 172.21.15.146:0/3528018339 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-04-25T10:58:54.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.983+0000 7f06e63d1640 1 --2- 172.21.15.146:0/3528018339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06e00c7540 0x7f06e00c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:54.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.983+0000 7f06e53cf640 1 -- 172.21.15.146:0/3528018339 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f06e00b1b20 msgr2=0x7f06e0151850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:54.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.983+0000 7f06e53cf640 1 --2- 172.21.15.146:0/3528018339 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f06e00b1b20 0x7f06e0151850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:54.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.983+0000 7f06e53cf640 1 -- 172.21.15.146:0/3528018339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06e00c7540 msgr2=0x7f06e00c7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:54.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.983+0000 7f06e53cf640 1 --2- 172.21.15.146:0/3528018339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06e00c7540 0x7f06e00c7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:54.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.983+0000 7f06e53cf640 1 -- 172.21.15.146:0/3528018339 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e0152680 con 0x7f06e0151e10 2024-04-25T10:58:54.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.983+0000 7f06e5bd0640 1 --2- 172.21.15.146:0/3528018339 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f06e00b1b20 0x7f06e0151850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:58:54.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.983+0000 7f06e53cf640 1 --2- 172.21.15.146:0/3528018339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06e0151e10 0x7f06e014f8a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f06dc00c440 tx=0x7f06dc00c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:54.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.983+0000 7f06d67fc640 1 -- 172.21.15.146:0/3528018339 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc0052d0 con 0x7f06e0151e10 2024-04-25T10:58:54.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.983+0000 7f06d67fc640 1 -- 172.21.15.146:0/3528018339 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06dc005470 con 0x7f06e0151e10 2024-04-25T10:58:54.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.983+0000 7f06e7e34640 1 -- 172.21.15.146:0/3528018339 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06e0157a70 con 0x7f06e0151e10 2024-04-25T10:58:54.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.983+0000 7f06d67fc640 1 -- 172.21.15.146:0/3528018339 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc007260 con 0x7f06e0151e10 2024-04-25T10:58:54.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.983+0000 7f06e7e34640 1 -- 172.21.15.146:0/3528018339 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06e00c6d40 con 0x7f06e0151e10 2024-04-25T10:58:54.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.983+0000 7f06d67fc640 1 -- 172.21.15.146:0/3528018339 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f06dc0074c0 con 0x7f06e0151e10 2024-04-25T10:58:54.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.983+0000 7f06d67fc640 1 -- 172.21.15.146:0/3528018339 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f06dc003b70 con 0x7f06e0151e10 2024-04-25T10:58:54.987 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.984+0000 7f06e7e34640 1 -- 172.21.15.146:0/3528018339 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06e0050740 con 0x7f06e0151e10 2024-04-25T10:58:54.988 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:54.987+0000 7f06d67fc640 1 -- 172.21.15.146:0/3528018339 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f06dc020070 con 0x7f06e0151e10 2024-04-25T10:58:55.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.059+0000 7f06e7e34640 1 -- 172.21.15.146:0/3528018339 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0c380208-30e4-47a1-83b0-423ae5ead112", "id": 0} v 0) -- 0x7f06e0158e00 con 0x7f06e0151e10 2024-04-25T10:58:55.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.063+0000 7f06d67fc640 1 -- 172.21.15.146:0/3528018339 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0c380208-30e4-47a1-83b0-423ae5ead112", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f06dc0035f0 con 0x7f06e0151e10 2024-04-25T10:58:55.064 INFO:teuthology.orchestra.run.smithi146.stdout:0 2024-04-25T10:58:55.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.064+0000 7f06e7e34640 1 -- 172.21.15.146:0/3528018339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06e0151e10 msgr2=0x7f06e014f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:55.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.064+0000 7f06e7e34640 1 --2- 172.21.15.146:0/3528018339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06e0151e10 0x7f06e014f8a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f06dc00c440 tx=0x7f06dc00c900 comp rx=0 tx=0).stop 2024-04-25T10:58:55.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.064+0000 7f06e7e34640 1 -- 172.21.15.146:0/3528018339 shutdown_connections 2024-04-25T10:58:55.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.064+0000 7f06e7e34640 1 --2- 172.21.15.146:0/3528018339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06e00c7540 0x7f06e00c7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.064+0000 7f06e7e34640 1 --2- 172.21.15.146:0/3528018339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06e0151e10 0x7f06e014f8a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.064+0000 7f06e7e34640 1 --2- 172.21.15.146:0/3528018339 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f06e00b1b20 0x7f06e0151850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.064+0000 7f06e7e34640 1 -- 172.21.15.146:0/3528018339 >> 172.21.15.146:0/3528018339 conn(0x7f06e009b7d0 msgr2=0x7f06e0070b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:55.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.064+0000 7f06e7e34640 1 -- 172.21.15.146:0/3528018339 shutdown_connections 2024-04-25T10:58:55.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.064+0000 7f06e7e34640 1 -- 172.21.15.146:0/3528018339 wait complete. 2024-04-25T10:58:55.075 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new b187c42c-5042-4ecd-9da5-f6b00cb9348e 1 2024-04-25T10:58:55.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.222+0000 7f397eed8640 1 Processor -- start 2024-04-25T10:58:55.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.223+0000 7f397eed8640 1 -- start start 2024-04-25T10:58:55.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.224+0000 7f397eed8640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f39780b38f0 0x7f3978160460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:55.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.224+0000 7f397eed8640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f39780b7760 0x7f397806d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:55.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.224+0000 7f397eed8640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39781609a0 con 0x7f39780b1210 2024-04-25T10:58:55.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.224+0000 7f397eed8640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f397806d500 con 0x7f39780b7760 2024-04-25T10:58:55.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.224+0000 7f397eed8640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f397806c8f0 con 0x7f39780b38f0 2024-04-25T10:58:55.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.224+0000 7f397cc74640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f39780b38f0 0x7f3978160460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:55.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.224+0000 7f397d475640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f39780b1210 0x7f39780b1620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:55406/0 (socket says 172.21.15.146:55406) 2024-04-25T10:58:55.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.224+0000 7f397d475640 1 -- 172.21.15.146:0/3712114111 learned_addr learned my addr 172.21.15.146:0/3712114111 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-04-25T10:58:55.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.224+0000 7f3977fff640 1 --2- 172.21.15.146:0/3712114111 >> v2:172.21.15.69:3300/0 conn(0x7f39780b7760 0x7f397806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:55.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.224+0000 7f397cc74640 1 -- 172.21.15.146:0/3712114111 >> v2:172.21.15.69:3300/0 conn(0x7f39780b7760 msgr2=0x7f397806d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:55.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.224+0000 7f397cc74640 1 --2- 172.21.15.146:0/3712114111 >> v2:172.21.15.69:3300/0 conn(0x7f39780b7760 0x7f397806d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.224+0000 7f397cc74640 1 -- 172.21.15.146:0/3712114111 >> v1:172.21.15.146:6789/0 conn(0x7f39780b1210 legacy=0x7f39780b1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:55.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.224+0000 7f397cc74640 1 -- 172.21.15.146:0/3712114111 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f397806cbd0 con 0x7f39780b38f0 2024-04-25T10:58:55.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.224+0000 7f39777fe640 1 -- 172.21.15.146:0/3712114111 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 854907522 0 0) 0x7f39781609a0 con 0x7f39780b1210 2024-04-25T10:58:55.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.225+0000 7f397cc74640 1 --2- 172.21.15.146:0/3712114111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39780b38f0 0x7f3978160460 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f397809bc50 tx=0x7f396c02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8aa18cec3dd1d9f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:55.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.225+0000 7f39777fe640 1 -- 172.21.15.146:0/3712114111 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c037070 con 0x7f39780b38f0 2024-04-25T10:58:55.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.225+0000 7f39777fe640 1 -- 172.21.15.146:0/3712114111 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f396c032440 con 0x7f39780b38f0 2024-04-25T10:58:55.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.225+0000 7f39777fe640 1 -- 172.21.15.146:0/3712114111 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c032810 con 0x7f39780b38f0 2024-04-25T10:58:55.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.226+0000 7f397eed8640 1 -- 172.21.15.146:0/3712114111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39780b38f0 msgr2=0x7f3978160460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:55.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.226+0000 7f397eed8640 1 --2- 172.21.15.146:0/3712114111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39780b38f0 0x7f3978160460 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f397809bc50 tx=0x7f396c02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.226+0000 7f397eed8640 1 -- 172.21.15.146:0/3712114111 shutdown_connections 2024-04-25T10:58:55.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.226+0000 7f397eed8640 1 --2- 172.21.15.146:0/3712114111 >> v2:172.21.15.69:3300/0 conn(0x7f39780b7760 0x7f397806d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.226+0000 7f397eed8640 1 --2- 172.21.15.146:0/3712114111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39780b38f0 0x7f3978160460 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.226+0000 7f397eed8640 1 -- 172.21.15.146:0/3712114111 >> 172.21.15.146:0/3712114111 conn(0x7f397809b7d0 msgr2=0x7f39780b7050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:55.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.226+0000 7f397eed8640 1 -- 172.21.15.146:0/3712114111 shutdown_connections 2024-04-25T10:58:55.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.226+0000 7f397eed8640 1 -- 172.21.15.146:0/3712114111 wait complete. 2024-04-25T10:58:55.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.228+0000 7f397eed8640 1 Processor -- start 2024-04-25T10:58:55.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.228+0000 7f397eed8640 1 -- start start 2024-04-25T10:58:55.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.229+0000 7f397eed8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f39780b1210 0x7f39780ce640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:55.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.229+0000 7f397eed8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39780b38f0 0x7f39780c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:55.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.229+0000 7f397eed8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39780b7760 0x7f39780c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:55.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.229+0000 7f397eed8640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f397806c5c0 con 0x7f39780b38f0 2024-04-25T10:58:55.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.229+0000 7f397eed8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f397806d190 con 0x7f39780b7760 2024-04-25T10:58:55.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.229+0000 7f397eed8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f397806ded0 con 0x7f39780b1210 2024-04-25T10:58:55.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.229+0000 7f397d475640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39780b7760 0x7f39780c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:55.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.229+0000 7f397cc74640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f39780b1210 0x7f39780ce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:55.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.230+0000 7f397d475640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39780b7760 0x7f39780c9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57330/0 (socket says 172.21.15.146:57330) 2024-04-25T10:58:55.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.230+0000 7f397cc74640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f39780b1210 0x7f39780ce640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:43972/0 (socket says 172.21.15.146:43972) 2024-04-25T10:58:55.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.230+0000 7f397d475640 1 -- 172.21.15.146:0/2903236167 learned_addr learned my addr 172.21.15.146:0/2903236167 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-04-25T10:58:55.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.230+0000 7f3977fff640 1 --2- 172.21.15.146:0/2903236167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39780b38f0 0x7f39780c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:55.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.230+0000 7f397d475640 1 -- 172.21.15.146:0/2903236167 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f39780b1210 msgr2=0x7f39780ce640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:55.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.230+0000 7f397d475640 1 --2- 172.21.15.146:0/2903236167 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f39780b1210 0x7f39780ce640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.230+0000 7f397d475640 1 -- 172.21.15.146:0/2903236167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39780b38f0 msgr2=0x7f39780c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:55.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.230+0000 7f397d475640 1 --2- 172.21.15.146:0/2903236167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39780b38f0 0x7f39780c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.230+0000 7f397d475640 1 -- 172.21.15.146:0/2903236167 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39780c7660 con 0x7f39780b7760 2024-04-25T10:58:55.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.230+0000 7f3977fff640 1 --2- 172.21.15.146:0/2903236167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39780b38f0 0x7f39780c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:58:55.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.230+0000 7f397cc74640 1 --2- 172.21.15.146:0/2903236167 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f39780b1210 0x7f39780ce640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:58:55.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.230+0000 7f397d475640 1 --2- 172.21.15.146:0/2903236167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39780b7760 0x7f39780c9490 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f397000c700 tx=0x7f397000cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:55.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.230+0000 7f39757fa640 1 -- 172.21.15.146:0/2903236167 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3970018070 con 0x7f39780b7760 2024-04-25T10:58:55.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.230+0000 7f39757fa640 1 -- 172.21.15.146:0/2903236167 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f397000ce10 con 0x7f39780b7760 2024-04-25T10:58:55.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.230+0000 7f397eed8640 1 -- 172.21.15.146:0/2903236167 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39780c7940 con 0x7f39780b7760 2024-04-25T10:58:55.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.230+0000 7f39757fa640 1 -- 172.21.15.146:0/2903236167 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397000a880 con 0x7f39780b7760 2024-04-25T10:58:55.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.230+0000 7f397eed8640 1 -- 172.21.15.146:0/2903236167 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39780c7f00 con 0x7f39780b7760 2024-04-25T10:58:55.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.230+0000 7f39757fa640 1 -- 172.21.15.146:0/2903236167 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f397001b070 con 0x7f39780b7760 2024-04-25T10:58:55.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.230+0000 7f39757fa640 1 -- 172.21.15.146:0/2903236167 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f397001d7f0 con 0x7f39780b7760 2024-04-25T10:58:55.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.231+0000 7f397eed8640 1 -- 172.21.15.146:0/2903236167 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3978050740 con 0x7f39780b7760 2024-04-25T10:58:55.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.234+0000 7f39757fa640 1 -- 172.21.15.146:0/2903236167 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f39700078b0 con 0x7f39780b7760 2024-04-25T10:58:55.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.308+0000 7f397eed8640 1 -- 172.21.15.146:0/2903236167 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b187c42c-5042-4ecd-9da5-f6b00cb9348e", "id": 1} v 0) -- 0x7f39780cfa60 con 0x7f39780b7760 2024-04-25T10:58:55.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.311+0000 7f39757fa640 1 -- 172.21.15.146:0/2903236167 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b187c42c-5042-4ecd-9da5-f6b00cb9348e", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f39700071a0 con 0x7f39780b7760 2024-04-25T10:58:55.312 INFO:teuthology.orchestra.run.smithi146.stdout:1 2024-04-25T10:58:55.314 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.313+0000 7f397eed8640 1 -- 172.21.15.146:0/2903236167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39780b7760 msgr2=0x7f39780c9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:55.314 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.313+0000 7f397eed8640 1 --2- 172.21.15.146:0/2903236167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39780b7760 0x7f39780c9490 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f397000c700 tx=0x7f397000cbc0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.314 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.313+0000 7f397eed8640 1 -- 172.21.15.146:0/2903236167 shutdown_connections 2024-04-25T10:58:55.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.313+0000 7f397eed8640 1 --2- 172.21.15.146:0/2903236167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39780b7760 0x7f39780c9490 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.313+0000 7f397eed8640 1 --2- 172.21.15.146:0/2903236167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39780b38f0 0x7f39780c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.313+0000 7f397eed8640 1 --2- 172.21.15.146:0/2903236167 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f39780b1210 0x7f39780ce640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.313+0000 7f397eed8640 1 -- 172.21.15.146:0/2903236167 >> 172.21.15.146:0/2903236167 conn(0x7f397809b7d0 msgr2=0x7f397815bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:55.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.313+0000 7f397eed8640 1 -- 172.21.15.146:0/2903236167 shutdown_connections 2024-04-25T10:58:55.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.313+0000 7f397eed8640 1 -- 172.21.15.146:0/2903236167 wait complete. 2024-04-25T10:58:55.328 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new 7accb18d-5798-4aca-a73c-587a47c0d982 2 2024-04-25T10:58:55.435 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.435+0000 7f218a020640 1 Processor -- start 2024-04-25T10:58:55.436 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.435+0000 7f218a020640 1 -- start start 2024-04-25T10:58:55.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.436+0000 7f218a020640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f218415e170 0x7f21840b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:55.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.436+0000 7f218a020640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f218415d240 0x7f218406d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:55.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.436+0000 7f218a020640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f218415e650 con 0x7f21840b7760 2024-04-25T10:58:55.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.436+0000 7f218a020640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f21840cf320 con 0x7f218415d240 2024-04-25T10:58:55.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.436+0000 7f218a020640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f21840cf600 con 0x7f218415e170 2024-04-25T10:58:55.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.436+0000 7f21837fe640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f218415e170 0x7f21840b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:55.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.436+0000 7f2183fff640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f21840b7760 0x7f21840b7b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:55418/0 (socket says 172.21.15.146:55418) 2024-04-25T10:58:55.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.436+0000 7f2183fff640 1 -- 172.21.15.146:0/3180265824 learned_addr learned my addr 172.21.15.146:0/3180265824 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-04-25T10:58:55.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.436+0000 7f2182ffd640 1 --2- 172.21.15.146:0/3180265824 >> v2:172.21.15.69:3300/0 conn(0x7f218415d240 0x7f218406d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:55.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.437+0000 7f21837fe640 1 -- 172.21.15.146:0/3180265824 >> v2:172.21.15.69:3300/0 conn(0x7f218415d240 msgr2=0x7f218406d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:55.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.437+0000 7f21837fe640 1 --2- 172.21.15.146:0/3180265824 >> v2:172.21.15.69:3300/0 conn(0x7f218415d240 0x7f218406d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.437+0000 7f21837fe640 1 -- 172.21.15.146:0/3180265824 >> v1:172.21.15.146:6789/0 conn(0x7f21840b7760 legacy=0x7f21840b7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:55.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.437+0000 7f21827fc640 1 -- 172.21.15.146:0/3180265824 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2948088027 0 0) 0x7f218415e650 con 0x7f21840b7760 2024-04-25T10:58:55.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.437+0000 7f21837fe640 1 -- 172.21.15.146:0/3180265824 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21840cf8e0 con 0x7f218415e170 2024-04-25T10:58:55.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.438+0000 7f21837fe640 1 --2- 172.21.15.146:0/3180265824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f218415e170 0x7f21840b3d70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2184072b30 tx=0x7f217402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=822a8ab3e5484372 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:55.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.438+0000 7f21827fc640 1 -- 172.21.15.146:0/3180265824 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2174037070 con 0x7f218415e170 2024-04-25T10:58:55.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.438+0000 7f21827fc640 1 -- 172.21.15.146:0/3180265824 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2174032440 con 0x7f218415e170 2024-04-25T10:58:55.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.438+0000 7f21827fc640 1 -- 172.21.15.146:0/3180265824 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2174032810 con 0x7f218415e170 2024-04-25T10:58:55.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.439+0000 7f218a020640 1 -- 172.21.15.146:0/3180265824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f218415e170 msgr2=0x7f21840b3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:55.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.439+0000 7f218a020640 1 --2- 172.21.15.146:0/3180265824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f218415e170 0x7f21840b3d70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2184072b30 tx=0x7f217402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.439+0000 7f218a020640 1 -- 172.21.15.146:0/3180265824 shutdown_connections 2024-04-25T10:58:55.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.439+0000 7f218a020640 1 --2- 172.21.15.146:0/3180265824 >> v2:172.21.15.69:3300/0 conn(0x7f218415d240 0x7f218406d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.439+0000 7f218a020640 1 --2- 172.21.15.146:0/3180265824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f218415e170 0x7f21840b3d70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.439+0000 7f218a020640 1 -- 172.21.15.146:0/3180265824 >> 172.21.15.146:0/3180265824 conn(0x7f218409b7d0 msgr2=0x7f218409bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:55.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.439+0000 7f218a020640 1 -- 172.21.15.146:0/3180265824 shutdown_connections 2024-04-25T10:58:55.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.439+0000 7f218a020640 1 -- 172.21.15.146:0/3180265824 wait complete. 2024-04-25T10:58:55.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.441+0000 7f218a020640 1 Processor -- start 2024-04-25T10:58:55.441 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.441+0000 7f218a020640 1 -- start start 2024-04-25T10:58:55.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.442+0000 7f218a020640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21840b7760 0x7f21840af850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:55.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.442+0000 7f218a020640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f218415d240 0x7f218414bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:55.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.442+0000 7f218a020640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f218415e170 0x7f21841f21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:55.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.442+0000 7f218a020640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f218406dc10 con 0x7f21840b7760 2024-04-25T10:58:55.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.442+0000 7f218a020640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f21840b42b0 con 0x7f218415d240 2024-04-25T10:58:55.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.442+0000 7f218a020640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f218406dd90 con 0x7f218415e170 2024-04-25T10:58:55.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.442+0000 7f2182ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f218415d240 0x7f218414bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:55.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.442+0000 7f2183fff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f218415e170 0x7f21841f21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:55.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.442+0000 7f2182ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f218415d240 0x7f218414bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57362/0 (socket says 172.21.15.146:57362) 2024-04-25T10:58:55.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.442+0000 7f2182ffd640 1 -- 172.21.15.146:0/66353648 learned_addr learned my addr 172.21.15.146:0/66353648 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-04-25T10:58:55.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.442+0000 7f21837fe640 1 --2- 172.21.15.146:0/66353648 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21840b7760 0x7f21840af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:55.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.443+0000 7f2183fff640 1 -- 172.21.15.146:0/66353648 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f218415d240 msgr2=0x7f218414bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:55.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.443+0000 7f2183fff640 1 --2- 172.21.15.146:0/66353648 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f218415d240 0x7f218414bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.443+0000 7f2183fff640 1 -- 172.21.15.146:0/66353648 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21840b7760 msgr2=0x7f21840af850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:55.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.443+0000 7f2183fff640 1 --2- 172.21.15.146:0/66353648 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21840b7760 0x7f21840af850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.443+0000 7f2183fff640 1 -- 172.21.15.146:0/66353648 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21840d2110 con 0x7f218415e170 2024-04-25T10:58:55.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.443+0000 7f2183fff640 1 --2- 172.21.15.146:0/66353648 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f218415e170 0x7f21841f21f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2184072c20 tx=0x7f217c00cbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:55.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.443+0000 7f216bfff640 1 -- 172.21.15.146:0/66353648 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217c018070 con 0x7f218415e170 2024-04-25T10:58:55.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.443+0000 7f216bfff640 1 -- 172.21.15.146:0/66353648 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f217c004d60 con 0x7f218415e170 2024-04-25T10:58:55.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.443+0000 7f218a020640 1 -- 172.21.15.146:0/66353648 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21840d23f0 con 0x7f218415e170 2024-04-25T10:58:55.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.443+0000 7f216bfff640 1 -- 172.21.15.146:0/66353648 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217c00a880 con 0x7f218415e170 2024-04-25T10:58:55.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.443+0000 7f218a020640 1 -- 172.21.15.146:0/66353648 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f21840d29b0 con 0x7f218415e170 2024-04-25T10:58:55.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.443+0000 7f216bfff640 1 -- 172.21.15.146:0/66353648 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f217c01b070 con 0x7f218415e170 2024-04-25T10:58:55.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.443+0000 7f216bfff640 1 -- 172.21.15.146:0/66353648 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f217c012400 con 0x7f218415e170 2024-04-25T10:58:55.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.444+0000 7f218a020640 1 -- 172.21.15.146:0/66353648 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2184050740 con 0x7f218415e170 2024-04-25T10:58:55.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.447+0000 7f216bfff640 1 -- 172.21.15.146:0/66353648 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f217c007760 con 0x7f218415e170 2024-04-25T10:58:55.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.518+0000 7f218a020640 1 -- 172.21.15.146:0/66353648 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7accb18d-5798-4aca-a73c-587a47c0d982", "id": 2} v 0) -- 0x7f21840b8530 con 0x7f218415e170 2024-04-25T10:58:55.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.524+0000 7f216bfff640 1 -- 172.21.15.146:0/66353648 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7accb18d-5798-4aca-a73c-587a47c0d982", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f217c0070d0 con 0x7f218415e170 2024-04-25T10:58:55.524 INFO:teuthology.orchestra.run.smithi146.stdout:2 2024-04-25T10:58:55.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.525+0000 7f218a020640 1 -- 172.21.15.146:0/66353648 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f218415e170 msgr2=0x7f21841f21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:55.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.525+0000 7f218a020640 1 --2- 172.21.15.146:0/66353648 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f218415e170 0x7f21841f21f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2184072c20 tx=0x7f217c00cbe0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.525+0000 7f218a020640 1 -- 172.21.15.146:0/66353648 shutdown_connections 2024-04-25T10:58:55.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.525+0000 7f218a020640 1 --2- 172.21.15.146:0/66353648 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f218415e170 0x7f21841f21f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.525+0000 7f218a020640 1 --2- 172.21.15.146:0/66353648 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f218415d240 0x7f218414bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.525+0000 7f218a020640 1 --2- 172.21.15.146:0/66353648 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21840b7760 0x7f21840af850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.525+0000 7f218a020640 1 -- 172.21.15.146:0/66353648 >> 172.21.15.146:0/66353648 conn(0x7f218409b7d0 msgr2=0x7f218406c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:55.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.525+0000 7f218a020640 1 -- 172.21.15.146:0/66353648 shutdown_connections 2024-04-25T10:58:55.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.525+0000 7f218a020640 1 -- 172.21.15.146:0/66353648 wait complete. 2024-04-25T10:58:55.536 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new fd51970a-0370-46cc-afbe-ad075f94285d 3 2024-04-25T10:58:55.627 INFO:tasks.ceph.mgr.y.smithi069.stderr:2024-04-25T10:58:55.626+0000 7fecc21ee040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:58:55.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.645+0000 7f856580c640 1 Processor -- start 2024-04-25T10:58:55.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.645+0000 7f856580c640 1 -- start start 2024-04-25T10:58:55.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.646+0000 7f856580c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f85600ced00 0x7f85600cf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:55.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.646+0000 7f856580c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f85600b23c0 0x7f85600b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:55.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.646+0000 7f856580c640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85600cd180 con 0x7f8560160870 2024-04-25T10:58:55.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.646+0000 7f856580c640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f85600ce930 con 0x7f85600b23c0 2024-04-25T10:58:55.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.646+0000 7f856580c640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f85600cd460 con 0x7f85600ced00 2024-04-25T10:58:55.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.646+0000 7f855e7fc640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f85600ced00 0x7f85600cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:55.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.646+0000 7f855e7fc640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f85600ced00 0x7f85600cf110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57378/0 (socket says 172.21.15.146:57378) 2024-04-25T10:58:55.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.646+0000 7f855e7fc640 1 -- 172.21.15.146:0/3672457932 learned_addr learned my addr 172.21.15.146:0/3672457932 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-04-25T10:58:55.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.646+0000 7f855effd640 1 --1- 172.21.15.146:0/3672457932 >> v1:172.21.15.146:6789/0 conn(0x7f8560160870 0x7f85600ce220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:55430/0 (socket says 172.21.15.146:55430) 2024-04-25T10:58:55.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.646+0000 7f855f7fe640 1 --2- 172.21.15.146:0/3672457932 >> v2:172.21.15.69:3300/0 conn(0x7f85600b23c0 0x7f85600b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:55.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.646+0000 7f855e7fc640 1 -- 172.21.15.146:0/3672457932 >> v2:172.21.15.69:3300/0 conn(0x7f85600b23c0 msgr2=0x7f85600b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:55.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.646+0000 7f855e7fc640 1 --2- 172.21.15.146:0/3672457932 >> v2:172.21.15.69:3300/0 conn(0x7f85600b23c0 0x7f85600b27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.647+0000 7f855e7fc640 1 -- 172.21.15.146:0/3672457932 >> v1:172.21.15.146:6789/0 conn(0x7f8560160870 legacy=0x7f85600ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:55.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.647+0000 7f855dffb640 1 -- 172.21.15.146:0/3672457932 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 79201018 0 0) 0x7f85600cd180 con 0x7f8560160870 2024-04-25T10:58:55.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.647+0000 7f855e7fc640 1 -- 172.21.15.146:0/3672457932 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85600cd740 con 0x7f85600ced00 2024-04-25T10:58:55.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.647+0000 7f855f7fe640 1 --2- 172.21.15.146:0/3672457932 >> v2:172.21.15.69:3300/0 conn(0x7f85600b23c0 0x7f85600b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:58:55.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.647+0000 7f855e7fc640 1 --2- 172.21.15.146:0/3672457932 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85600ced00 0x7f85600cf110 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8560073720 tx=0x7f855002a150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=827fee44afea3eb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:55.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.647+0000 7f855dffb640 1 -- 172.21.15.146:0/3672457932 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8550037070 con 0x7f85600ced00 2024-04-25T10:58:55.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.647+0000 7f855dffb640 1 -- 172.21.15.146:0/3672457932 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8550032440 con 0x7f85600ced00 2024-04-25T10:58:55.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.647+0000 7f855dffb640 1 -- 172.21.15.146:0/3672457932 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8550032810 con 0x7f85600ced00 2024-04-25T10:58:55.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.648+0000 7f856580c640 1 -- 172.21.15.146:0/3672457932 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85600ced00 msgr2=0x7f85600cf110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:55.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.648+0000 7f856580c640 1 --2- 172.21.15.146:0/3672457932 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85600ced00 0x7f85600cf110 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8560073720 tx=0x7f855002a150 comp rx=0 tx=0).stop 2024-04-25T10:58:55.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.648+0000 7f856580c640 1 -- 172.21.15.146:0/3672457932 shutdown_connections 2024-04-25T10:58:55.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.648+0000 7f856580c640 1 --2- 172.21.15.146:0/3672457932 >> v2:172.21.15.69:3300/0 conn(0x7f85600b23c0 0x7f85600b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.648+0000 7f856580c640 1 --2- 172.21.15.146:0/3672457932 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85600ced00 0x7f85600cf110 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.648+0000 7f856580c640 1 -- 172.21.15.146:0/3672457932 >> 172.21.15.146:0/3672457932 conn(0x7f856009b7d0 msgr2=0x7f856009bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:55.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.648+0000 7f856580c640 1 -- 172.21.15.146:0/3672457932 shutdown_connections 2024-04-25T10:58:55.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.648+0000 7f856580c640 1 -- 172.21.15.146:0/3672457932 wait complete. 2024-04-25T10:58:55.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.650+0000 7f856580c640 1 Processor -- start 2024-04-25T10:58:55.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.650+0000 7f856580c640 1 -- start start 2024-04-25T10:58:55.654 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.652+0000 7f856580c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85600b23c0 0x7f85600c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:55.654 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.652+0000 7f856580c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85600ced00 0x7f8560071610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:55.654 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.652+0000 7f856580c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8560160870 0x7f8560071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:55.654 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.652+0000 7f856580c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f85600cfa70 con 0x7f85600b23c0 2024-04-25T10:58:55.654 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.652+0000 7f856580c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f85600b2d10 con 0x7f85600ced00 2024-04-25T10:58:55.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.652+0000 7f856580c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f85600d1cb0 con 0x7f8560160870 2024-04-25T10:58:55.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.652+0000 7f855e7fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85600ced00 0x7f8560071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:55.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.652+0000 7f855f7fe640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8560160870 0x7f8560071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:55.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.652+0000 7f855e7fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85600ced00 0x7f8560071610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57382/0 (socket says 172.21.15.146:57382) 2024-04-25T10:58:55.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.652+0000 7f855f7fe640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8560160870 0x7f8560071b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:43976/0 (socket says 172.21.15.146:43976) 2024-04-25T10:58:55.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.652+0000 7f855e7fc640 1 -- 172.21.15.146:0/3628709347 learned_addr learned my addr 172.21.15.146:0/3628709347 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-04-25T10:58:55.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.652+0000 7f855effd640 1 --2- 172.21.15.146:0/3628709347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85600b23c0 0x7f85600c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:55.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.652+0000 7f855f7fe640 1 -- 172.21.15.146:0/3628709347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85600ced00 msgr2=0x7f8560071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:55.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.652+0000 7f855f7fe640 1 --2- 172.21.15.146:0/3628709347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85600ced00 0x7f8560071610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.652+0000 7f855f7fe640 1 -- 172.21.15.146:0/3628709347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85600b23c0 msgr2=0x7f85600c9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:55.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.652+0000 7f855f7fe640 1 --2- 172.21.15.146:0/3628709347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85600b23c0 0x7f85600c9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.652+0000 7f855f7fe640 1 -- 172.21.15.146:0/3628709347 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85600ca350 con 0x7f8560160870 2024-04-25T10:58:55.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.652+0000 7f855e7fc640 1 --2- 172.21.15.146:0/3628709347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85600ced00 0x7f8560071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:58:55.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.652+0000 7f855f7fe640 1 --2- 172.21.15.146:0/3628709347 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8560160870 0x7f8560071b50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8558007d70 tx=0x7f855800d4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:55.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.652+0000 7f8547fff640 1 -- 172.21.15.146:0/3628709347 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855801a070 con 0x7f8560160870 2024-04-25T10:58:55.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.652+0000 7f8547fff640 1 -- 172.21.15.146:0/3628709347 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8558003990 con 0x7f8560160870 2024-04-25T10:58:55.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.652+0000 7f856580c640 1 -- 172.21.15.146:0/3628709347 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85600ca630 con 0x7f8560160870 2024-04-25T10:58:55.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.653+0000 7f8547fff640 1 -- 172.21.15.146:0/3628709347 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8558003d80 con 0x7f8560160870 2024-04-25T10:58:55.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.653+0000 7f856580c640 1 -- 172.21.15.146:0/3628709347 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f85600cabf0 con 0x7f8560160870 2024-04-25T10:58:55.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.653+0000 7f8547fff640 1 -- 172.21.15.146:0/3628709347 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f855801d070 con 0x7f8560160870 2024-04-25T10:58:55.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.653+0000 7f8547fff640 1 -- 172.21.15.146:0/3628709347 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f85580158d0 con 0x7f8560160870 2024-04-25T10:58:55.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.653+0000 7f856580c640 1 -- 172.21.15.146:0/3628709347 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85600c9360 con 0x7f8560160870 2024-04-25T10:58:55.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.656+0000 7f8547fff640 1 -- 172.21.15.146:0/3628709347 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f8558024070 con 0x7f8560160870 2024-04-25T10:58:55.657 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T10:58:55.656+0000 7f53b8f19040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:58:55.713 INFO:tasks.ceph.mgr.y.smithi069.stderr:2024-04-25T10:58:55.713+0000 7fecc21ee040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:58:55.728 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.728+0000 7f856580c640 1 -- 172.21.15.146:0/3628709347 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd new", "uuid": "fd51970a-0370-46cc-afbe-ad075f94285d", "id": 3} v 0) -- 0x7f8560160370 con 0x7f8560160870 2024-04-25T10:58:55.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.733+0000 7f8547fff640 1 -- 172.21.15.146:0/3628709347 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fd51970a-0370-46cc-afbe-ad075f94285d", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f8558010070 con 0x7f8560160870 2024-04-25T10:58:55.734 INFO:teuthology.orchestra.run.smithi146.stdout:3 2024-04-25T10:58:55.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.734+0000 7f856580c640 1 -- 172.21.15.146:0/3628709347 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8560160870 msgr2=0x7f8560071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:55.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.734+0000 7f856580c640 1 --2- 172.21.15.146:0/3628709347 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8560160870 0x7f8560071b50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8558007d70 tx=0x7f855800d4a0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.734+0000 7f856580c640 1 -- 172.21.15.146:0/3628709347 shutdown_connections 2024-04-25T10:58:55.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.734+0000 7f856580c640 1 --2- 172.21.15.146:0/3628709347 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8560160870 0x7f8560071b50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.734+0000 7f856580c640 1 --2- 172.21.15.146:0/3628709347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85600ced00 0x7f8560071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.734+0000 7f856580c640 1 --2- 172.21.15.146:0/3628709347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85600b23c0 0x7f85600c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.734+0000 7f856580c640 1 -- 172.21.15.146:0/3628709347 >> 172.21.15.146:0/3628709347 conn(0x7f856009b7d0 msgr2=0x7f85600704b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:55.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.735+0000 7f856580c640 1 -- 172.21.15.146:0/3628709347 shutdown_connections 2024-04-25T10:58:55.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.735+0000 7f856580c640 1 -- 172.21.15.146:0/3628709347 wait complete. 2024-04-25T10:58:55.746 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new f4ad69ae-fef4-431d-bd99-d3eeada93c39 4 2024-04-25T10:58:55.749 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T10:58:55.749+0000 7f53b8f19040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:58:55.755 INFO:tasks.ceph.mgr.y.smithi069.stderr:2024-04-25T10:58:55.755+0000 7fecc21ee040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:58:55.795 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T10:58:55.795+0000 7f53b8f19040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:58:55.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.853+0000 7fc013605640 1 Processor -- start 2024-04-25T10:58:55.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.853+0000 7fc013605640 1 -- start start 2024-04-25T10:58:55.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.855+0000 7fc013605640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc00c161760 0x7fc00c0b7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:55.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.855+0000 7fc013605640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc00c0d2200 0x7fc00c0b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:55.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.855+0000 7fc013605640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc00c0cf4b0 con 0x7fc00c0b82d0 2024-04-25T10:58:55.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.855+0000 7fc013605640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fc00c0d00c0 con 0x7fc00c161760 2024-04-25T10:58:55.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.855+0000 7fc013605640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc00c0cf790 con 0x7fc00c0d2200 2024-04-25T10:58:55.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.855+0000 7fc0113a1640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc00c161760 0x7fc00c0b7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:55.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.855+0000 7fc0113a1640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc00c161760 0x7fc00c0b7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57392/0 (socket says 172.21.15.146:57392) 2024-04-25T10:58:55.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.855+0000 7fc0113a1640 1 -- 172.21.15.146:0/27500497 learned_addr learned my addr 172.21.15.146:0/27500497 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-04-25T10:58:55.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.855+0000 7fc011ba2640 1 --2- 172.21.15.146:0/27500497 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc00c0d2200 0x7fc00c0b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:55.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.855+0000 7fc0113a1640 1 -- 172.21.15.146:0/27500497 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc00c0d2200 msgr2=0x7fc00c0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:55.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.855+0000 7fc0113a1640 1 --2- 172.21.15.146:0/27500497 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc00c0d2200 0x7fc00c0b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.855+0000 7fc0113a1640 1 -- 172.21.15.146:0/27500497 >> v1:172.21.15.69:6789/0 conn(0x7fc00c0b82d0 legacy=0x7fc00c0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:55.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.855+0000 7fc0113a1640 1 -- 172.21.15.146:0/27500497 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00c0cfa70 con 0x7fc00c161760 2024-04-25T10:58:55.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.855+0000 7fc011ba2640 1 --2- 172.21.15.146:0/27500497 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc00c0d2200 0x7fc00c0b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:58:55.858 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.856+0000 7fc0113a1640 1 --2- 172.21.15.146:0/27500497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc00c161760 0x7fc00c0b7d90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009870 tx=0x7fbffc02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93fbccbfa0919131 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:55.858 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.856+0000 7fc003fff640 1 -- 172.21.15.146:0/27500497 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc037070 con 0x7fc00c161760 2024-04-25T10:58:55.858 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.856+0000 7fc003fff640 1 -- 172.21.15.146:0/27500497 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbffc032440 con 0x7fc00c161760 2024-04-25T10:58:55.858 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.856+0000 7fc003fff640 1 -- 172.21.15.146:0/27500497 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc032810 con 0x7fc00c161760 2024-04-25T10:58:55.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.857+0000 7fc013605640 1 -- 172.21.15.146:0/27500497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc00c161760 msgr2=0x7fc00c0b7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:55.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.857+0000 7fc013605640 1 --2- 172.21.15.146:0/27500497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc00c161760 0x7fc00c0b7d90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009870 tx=0x7fbffc02a270 comp rx=0 tx=0).stop 2024-04-25T10:58:55.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.857+0000 7fc013605640 1 -- 172.21.15.146:0/27500497 shutdown_connections 2024-04-25T10:58:55.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.857+0000 7fc013605640 1 --2- 172.21.15.146:0/27500497 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc00c0d2200 0x7fc00c0b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.857+0000 7fc013605640 1 --2- 172.21.15.146:0/27500497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc00c161760 0x7fc00c0b7d90 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.857+0000 7fc013605640 1 -- 172.21.15.146:0/27500497 >> 172.21.15.146:0/27500497 conn(0x7fc00c09b7d0 msgr2=0x7fc00c09bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:55.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.857+0000 7fc013605640 1 -- 172.21.15.146:0/27500497 shutdown_connections 2024-04-25T10:58:55.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.857+0000 7fc013605640 1 -- 172.21.15.146:0/27500497 wait complete. 2024-04-25T10:58:55.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.859+0000 7fc013605640 1 Processor -- start 2024-04-25T10:58:55.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.859+0000 7fc013605640 1 -- start start 2024-04-25T10:58:55.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.860+0000 7fc013605640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc00c0b82d0 0x7fc00c160450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:55.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.860+0000 7fc013605640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc00c0d2200 0x7fc00c09fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:55.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.860+0000 7fc013605640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc00c161760 0x7fc00c0a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:55.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.860+0000 7fc013605640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c0b7940 con 0x7fc00c161760 2024-04-25T10:58:55.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.860+0000 7fc013605640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c161b70 con 0x7fc00c0d2200 2024-04-25T10:58:55.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.860+0000 7fc013605640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc00c0b2920 con 0x7fc00c0b82d0 2024-04-25T10:58:55.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.860+0000 7fc0113a1640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc00c0b82d0 0x7fc00c160450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:55.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.860+0000 7fc010ba0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc00c0d2200 0x7fc00c09fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:55.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.860+0000 7fc0113a1640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc00c0b82d0 0x7fc00c160450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:43990/0 (socket says 172.21.15.146:43990) 2024-04-25T10:58:55.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.860+0000 7fc0113a1640 1 -- 172.21.15.146:0/157495069 learned_addr learned my addr 172.21.15.146:0/157495069 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-04-25T10:58:55.862 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.860+0000 7fc011ba2640 1 --2- 172.21.15.146:0/157495069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc00c161760 0x7fc00c0a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:55.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.860+0000 7fc0113a1640 1 -- 172.21.15.146:0/157495069 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc00c0d2200 msgr2=0x7fc00c09fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:55.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.860+0000 7fc0113a1640 1 --2- 172.21.15.146:0/157495069 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc00c0d2200 0x7fc00c09fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.860+0000 7fc0113a1640 1 -- 172.21.15.146:0/157495069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc00c161760 msgr2=0x7fc00c0a0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:55.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.860+0000 7fc0113a1640 1 --2- 172.21.15.146:0/157495069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc00c161760 0x7fc00c0a0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.860+0000 7fc0113a1640 1 -- 172.21.15.146:0/157495069 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00c09ca00 con 0x7fc00c0b82d0 2024-04-25T10:58:55.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.860+0000 7fc011ba2640 1 --2- 172.21.15.146:0/157495069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc00c161760 0x7fc00c0a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:58:55.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.861+0000 7fc0113a1640 1 --2- 172.21.15.146:0/157495069 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc00c0b82d0 0x7fc00c160450 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbffc000c00 tx=0x7fbffc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:55.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.861+0000 7fc001ffb640 1 -- 172.21.15.146:0/157495069 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc037040 con 0x7fc00c0b82d0 2024-04-25T10:58:55.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.861+0000 7fc001ffb640 1 -- 172.21.15.146:0/157495069 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbffc042070 con 0x7fc00c0b82d0 2024-04-25T10:58:55.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.861+0000 7fc013605640 1 -- 172.21.15.146:0/157495069 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc00c09cce0 con 0x7fc00c0b82d0 2024-04-25T10:58:55.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.861+0000 7fc001ffb640 1 -- 172.21.15.146:0/157495069 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc030880 con 0x7fc00c0b82d0 2024-04-25T10:58:55.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.861+0000 7fc013605640 1 -- 172.21.15.146:0/157495069 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc00c09d2a0 con 0x7fc00c0b82d0 2024-04-25T10:58:55.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.861+0000 7fc001ffb640 1 -- 172.21.15.146:0/157495069 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbffc030a20 con 0x7fc00c0b82d0 2024-04-25T10:58:55.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.861+0000 7fc001ffb640 1 -- 172.21.15.146:0/157495069 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fbffc03b430 con 0x7fc00c0b82d0 2024-04-25T10:58:55.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.862+0000 7fc013605640 1 -- 172.21.15.146:0/157495069 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc00c050740 con 0x7fc00c0b82d0 2024-04-25T10:58:55.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.864+0000 7fc001ffb640 1 -- 172.21.15.146:0/157495069 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fbffc047460 con 0x7fc00c0b82d0 2024-04-25T10:58:55.919 INFO:tasks.ceph.mgr.y.smithi069.stderr:2024-04-25T10:58:55.919+0000 7fecc21ee040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:58:55.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.936+0000 7fc013605640 1 -- 172.21.15.146:0/157495069 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f4ad69ae-fef4-431d-bd99-d3eeada93c39", "id": 4} v 0) -- 0x7fc00c06d100 con 0x7fc00c0b82d0 2024-04-25T10:58:55.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.942+0000 7fc001ffb640 1 -- 172.21.15.146:0/157495069 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f4ad69ae-fef4-431d-bd99-d3eeada93c39", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fbffc047640 con 0x7fc00c0b82d0 2024-04-25T10:58:55.943 INFO:teuthology.orchestra.run.smithi146.stdout:4 2024-04-25T10:58:55.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.943+0000 7fc013605640 1 -- 172.21.15.146:0/157495069 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc00c0b82d0 msgr2=0x7fc00c160450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:55.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.943+0000 7fc013605640 1 --2- 172.21.15.146:0/157495069 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc00c0b82d0 0x7fc00c160450 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbffc000c00 tx=0x7fbffc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.943+0000 7fc013605640 1 -- 172.21.15.146:0/157495069 shutdown_connections 2024-04-25T10:58:55.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.943+0000 7fc013605640 1 --2- 172.21.15.146:0/157495069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc00c161760 0x7fc00c0a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.943+0000 7fc013605640 1 --2- 172.21.15.146:0/157495069 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc00c0d2200 0x7fc00c09fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.943+0000 7fc013605640 1 --2- 172.21.15.146:0/157495069 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc00c0b82d0 0x7fc00c160450 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:55.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.943+0000 7fc013605640 1 -- 172.21.15.146:0/157495069 >> 172.21.15.146:0/157495069 conn(0x7fc00c09b7d0 msgr2=0x7fc00c06d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:55.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.943+0000 7fc013605640 1 -- 172.21.15.146:0/157495069 shutdown_connections 2024-04-25T10:58:55.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:55.943+0000 7fc013605640 1 -- 172.21.15.146:0/157495069 wait complete. 2024-04-25T10:58:55.954 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new c6370c19-9ca2-4590-92e0-2a174da6c834 5 2024-04-25T10:58:55.967 INFO:tasks.ceph.mgr.y.smithi069.stderr:2024-04-25T10:58:55.967+0000 7fecc21ee040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:58:55.974 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T10:58:55.973+0000 7f53b8f19040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:58:56.019 INFO:tasks.ceph.mgr.y.smithi069.stderr:2024-04-25T10:58:56.018+0000 7fecc21ee040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:58:56.024 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T10:58:56.024+0000 7f53b8f19040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:58:56.082 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.081+0000 7f8201c0c640 1 Processor -- start 2024-04-25T10:58:56.082 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.081+0000 7f8201c0c640 1 -- start start 2024-04-25T10:58:56.082 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T10:58:56.082+0000 7f53b8f19040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:58:56.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.083+0000 7f8201c0c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81fc0762d0 0x7f81fc0b1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:56.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.083+0000 7f8201c0c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f81fc163c60 0x7f81fc0b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:56.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.083+0000 7f8201c0c640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81fc0b77d0 con 0x7f81fc06ca10 2024-04-25T10:58:56.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.083+0000 7f8201c0c640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f81fc06c850 con 0x7f81fc163c60 2024-04-25T10:58:56.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.083+0000 7f8201c0c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f81fc0b3830 con 0x7f81fc0762d0 2024-04-25T10:58:56.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.083+0000 7f81fb7fe640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81fc0762d0 0x7f81fc0b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:56.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.083+0000 7f81fb7fe640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81fc0762d0 0x7f81fc0b1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:44004/0 (socket says 172.21.15.146:44004) 2024-04-25T10:58:56.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.083+0000 7f81fb7fe640 1 -- 172.21.15.146:0/3318531134 learned_addr learned my addr 172.21.15.146:0/3318531134 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-04-25T10:58:56.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.083+0000 7f81fbfff640 1 --2- 172.21.15.146:0/3318531134 >> v2:172.21.15.69:3300/0 conn(0x7f81fc163c60 0x7f81fc0b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:56.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.083+0000 7f81fb7fe640 1 -- 172.21.15.146:0/3318531134 >> v2:172.21.15.69:3300/0 conn(0x7f81fc163c60 msgr2=0x7f81fc0b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:56.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.083+0000 7f81fb7fe640 1 --2- 172.21.15.146:0/3318531134 >> v2:172.21.15.69:3300/0 conn(0x7f81fc163c60 0x7f81fc0b17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:56.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.083+0000 7f81fb7fe640 1 -- 172.21.15.146:0/3318531134 >> v1:172.21.15.69:6789/0 conn(0x7f81fc06ca10 legacy=0x7f81fc0cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:56.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.083+0000 7f81fb7fe640 1 -- 172.21.15.146:0/3318531134 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81fc0b7ab0 con 0x7f81fc0762d0 2024-04-25T10:58:56.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.084+0000 7f81fb7fe640 1 --2- 172.21.15.146:0/3318531134 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81fc0762d0 0x7f81fc0b1280 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f81f0009870 tx=0x7f81f002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff5a0df4e5303fe2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:56.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.084+0000 7f81fa7fc640 1 -- 172.21.15.146:0/3318531134 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f0037070 con 0x7f81fc0762d0 2024-04-25T10:58:56.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.084+0000 7f81fa7fc640 1 -- 172.21.15.146:0/3318531134 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81f0032440 con 0x7f81fc0762d0 2024-04-25T10:58:56.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.084+0000 7f81fa7fc640 1 -- 172.21.15.146:0/3318531134 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f0032810 con 0x7f81fc0762d0 2024-04-25T10:58:56.087 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.085+0000 7f8201c0c640 1 -- 172.21.15.146:0/3318531134 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81fc0762d0 msgr2=0x7f81fc0b1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:56.087 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.085+0000 7f8201c0c640 1 --2- 172.21.15.146:0/3318531134 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81fc0762d0 0x7f81fc0b1280 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f81f0009870 tx=0x7f81f002a270 comp rx=0 tx=0).stop 2024-04-25T10:58:56.087 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.085+0000 7f8201c0c640 1 -- 172.21.15.146:0/3318531134 shutdown_connections 2024-04-25T10:58:56.087 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.085+0000 7f8201c0c640 1 --2- 172.21.15.146:0/3318531134 >> v2:172.21.15.69:3300/0 conn(0x7f81fc163c60 0x7f81fc0b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:56.087 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.085+0000 7f8201c0c640 1 --2- 172.21.15.146:0/3318531134 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81fc0762d0 0x7f81fc0b1280 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:56.087 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.085+0000 7f8201c0c640 1 -- 172.21.15.146:0/3318531134 >> 172.21.15.146:0/3318531134 conn(0x7f81fc09b7d0 msgr2=0x7f81fc0b70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:56.087 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.085+0000 7f8201c0c640 1 -- 172.21.15.146:0/3318531134 shutdown_connections 2024-04-25T10:58:56.087 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.085+0000 7f8201c0c640 1 -- 172.21.15.146:0/3318531134 wait complete. 2024-04-25T10:58:56.088 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.087+0000 7f8201c0c640 1 Processor -- start 2024-04-25T10:58:56.088 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.087+0000 7f8201c0c640 1 -- start start 2024-04-25T10:58:56.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.088+0000 7f8201c0c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81fc06ca10 0x7f81fc09cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:56.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.088+0000 7f8201c0c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81fc0762d0 0x7f81fc154690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:56.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.088+0000 7f8201c0c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81fc163c60 0x7f81fc161a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:56.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.088+0000 7f8201c0c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f81fc0b2e90 con 0x7f81fc06ca10 2024-04-25T10:58:56.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.088+0000 7f8201c0c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f81fc0b34c0 con 0x7f81fc163c60 2024-04-25T10:58:56.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.088+0000 7f8201c0c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f81fc0cd850 con 0x7f81fc0762d0 2024-04-25T10:58:56.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.088+0000 7f81faffd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81fc0762d0 0x7f81fc154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:56.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.088+0000 7f81fbfff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81fc163c60 0x7f81fc161a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:56.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.088+0000 7f81faffd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81fc0762d0 0x7f81fc154690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:44010/0 (socket says 172.21.15.146:44010) 2024-04-25T10:58:56.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.088+0000 7f81faffd640 1 -- 172.21.15.146:0/1995337527 learned_addr learned my addr 172.21.15.146:0/1995337527 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-04-25T10:58:56.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.088+0000 7f81fb7fe640 1 --2- 172.21.15.146:0/1995337527 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81fc06ca10 0x7f81fc09cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:56.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.089+0000 7f81faffd640 1 -- 172.21.15.146:0/1995337527 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81fc163c60 msgr2=0x7f81fc161a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:56.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.089+0000 7f81faffd640 1 --2- 172.21.15.146:0/1995337527 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81fc163c60 0x7f81fc161a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:56.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.089+0000 7f81faffd640 1 -- 172.21.15.146:0/1995337527 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81fc06ca10 msgr2=0x7f81fc09cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:56.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.089+0000 7f81faffd640 1 --2- 172.21.15.146:0/1995337527 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81fc06ca10 0x7f81fc09cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:56.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.089+0000 7f81faffd640 1 -- 172.21.15.146:0/1995337527 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81fc09d290 con 0x7f81fc0762d0 2024-04-25T10:58:56.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.089+0000 7f81faffd640 1 --2- 172.21.15.146:0/1995337527 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81fc0762d0 0x7f81fc154690 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f81fc074a20 tx=0x7f81ec00c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:56.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.089+0000 7f81e3fff640 1 -- 172.21.15.146:0/1995337527 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec017070 con 0x7f81fc0762d0 2024-04-25T10:58:56.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.089+0000 7f81e3fff640 1 -- 172.21.15.146:0/1995337527 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81ec0052d0 con 0x7f81fc0762d0 2024-04-25T10:58:56.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.089+0000 7f81e3fff640 1 -- 172.21.15.146:0/1995337527 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec003810 con 0x7f81fc0762d0 2024-04-25T10:58:56.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.089+0000 7f8201c0c640 1 -- 172.21.15.146:0/1995337527 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81fc154bd0 con 0x7f81fc0762d0 2024-04-25T10:58:56.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.089+0000 7f8201c0c640 1 -- 172.21.15.146:0/1995337527 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f81fc0d01d0 con 0x7f81fc0762d0 2024-04-25T10:58:56.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.089+0000 7f81e3fff640 1 -- 172.21.15.146:0/1995337527 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f81ec01a070 con 0x7f81fc0762d0 2024-04-25T10:58:56.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.089+0000 7f81e3fff640 1 -- 172.21.15.146:0/1995337527 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f81ec011590 con 0x7f81fc0762d0 2024-04-25T10:58:56.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.090+0000 7f8201c0c640 1 -- 172.21.15.146:0/1995337527 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81fc050740 con 0x7f81fc0762d0 2024-04-25T10:58:56.093 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.092+0000 7f81e3fff640 1 -- 172.21.15.146:0/1995337527 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f81ec011ae0 con 0x7f81fc0762d0 2024-04-25T10:58:56.131 INFO:tasks.ceph.mgr.y.smithi069.stderr:2024-04-25T10:58:56.130+0000 7fecc21ee040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:58:56.166 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.165+0000 7f8201c0c640 1 -- 172.21.15.146:0/1995337527 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c6370c19-9ca2-4590-92e0-2a174da6c834", "id": 5} v 0) -- 0x7f81fc0cc2a0 con 0x7f81fc0762d0 2024-04-25T10:58:56.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.171+0000 7f81e3fff640 1 -- 172.21.15.146:0/1995337527 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c6370c19-9ca2-4590-92e0-2a174da6c834", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f81ec011cc0 con 0x7f81fc0762d0 2024-04-25T10:58:56.171 INFO:teuthology.orchestra.run.smithi146.stdout:5 2024-04-25T10:58:56.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.172+0000 7f8201c0c640 1 -- 172.21.15.146:0/1995337527 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81fc0762d0 msgr2=0x7f81fc154690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:56.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.172+0000 7f8201c0c640 1 --2- 172.21.15.146:0/1995337527 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81fc0762d0 0x7f81fc154690 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f81fc074a20 tx=0x7f81ec00c820 comp rx=0 tx=0).stop 2024-04-25T10:58:56.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.172+0000 7f8201c0c640 1 -- 172.21.15.146:0/1995337527 shutdown_connections 2024-04-25T10:58:56.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.172+0000 7f8201c0c640 1 --2- 172.21.15.146:0/1995337527 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81fc163c60 0x7f81fc161a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:56.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.172+0000 7f8201c0c640 1 --2- 172.21.15.146:0/1995337527 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81fc0762d0 0x7f81fc154690 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:56.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.172+0000 7f8201c0c640 1 --2- 172.21.15.146:0/1995337527 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81fc06ca10 0x7f81fc09cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:56.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.172+0000 7f8201c0c640 1 -- 172.21.15.146:0/1995337527 >> 172.21.15.146:0/1995337527 conn(0x7f81fc09b7d0 msgr2=0x7f81fc0cde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:56.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.172+0000 7f8201c0c640 1 -- 172.21.15.146:0/1995337527 shutdown_connections 2024-04-25T10:58:56.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.172+0000 7f8201c0c640 1 -- 172.21.15.146:0/1995337527 wait complete. 2024-04-25T10:58:56.183 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new e0e6a832-b1a1-42d7-836d-b4906e9860a0 6 2024-04-25T10:58:56.202 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T10:58:56.201+0000 7f53b8f19040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:58:56.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.289+0000 7faf57c25640 1 Processor -- start 2024-04-25T10:58:56.291 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.289+0000 7faf57c25640 1 -- start start 2024-04-25T10:58:56.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.291+0000 7faf57c25640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7faf500b1b20 0x7faf5006ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:56.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.291+0000 7faf57c25640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7faf500b7e70 0x7faf500d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:56.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.291+0000 7faf57c25640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf500cf960 con 0x7faf500b3d70 2024-04-25T10:58:56.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.291+0000 7faf57c25640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7faf5006d800 con 0x7faf500b1b20 2024-04-25T10:58:56.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.291+0000 7faf57c25640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7faf500b4230 con 0x7faf500b7e70 2024-04-25T10:58:56.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.291+0000 7faf561c2640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7faf500b7e70 0x7faf500d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:56.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.291+0000 7faf551c0640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7faf500b3d70 0x7faf500b7760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:55434/0 (socket says 172.21.15.146:55434) 2024-04-25T10:58:56.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.291+0000 7faf551c0640 1 -- 172.21.15.146:0/2402981647 learned_addr learned my addr 172.21.15.146:0/2402981647 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-04-25T10:58:56.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.291+0000 7faf559c1640 1 --2- 172.21.15.146:0/2402981647 >> v2:172.21.15.69:3300/0 conn(0x7faf500b1b20 0x7faf5006ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:56.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.291+0000 7faf561c2640 1 -- 172.21.15.146:0/2402981647 >> v2:172.21.15.69:3300/0 conn(0x7faf500b1b20 msgr2=0x7faf5006ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:58:56.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.291+0000 7faf561c2640 1 --2- 172.21.15.146:0/2402981647 >> v2:172.21.15.69:3300/0 conn(0x7faf500b1b20 0x7faf5006ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:56.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.291+0000 7faf561c2640 1 -- 172.21.15.146:0/2402981647 >> v1:172.21.15.146:6789/0 conn(0x7faf500b3d70 legacy=0x7faf500b7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:56.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.291+0000 7faf549bf640 1 -- 172.21.15.146:0/2402981647 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3560160174 0 0) 0x7faf500cf960 con 0x7faf500b3d70 2024-04-25T10:58:56.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.291+0000 7faf561c2640 1 -- 172.21.15.146:0/2402981647 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf500cf3a0 con 0x7faf500b7e70 2024-04-25T10:58:56.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.292+0000 7faf561c2640 1 --2- 172.21.15.146:0/2402981647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf500b7e70 0x7faf500d0900 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7faf50072b80 tx=0x7faf4802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b80cb9bbb6cc17e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:56.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.293+0000 7faf549bf640 1 -- 172.21.15.146:0/2402981647 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf48007740 con 0x7faf500b7e70 2024-04-25T10:58:56.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.293+0000 7faf549bf640 1 -- 172.21.15.146:0/2402981647 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf480078e0 con 0x7faf500b7e70 2024-04-25T10:58:56.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.293+0000 7faf549bf640 1 -- 172.21.15.146:0/2402981647 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf48007cb0 con 0x7faf500b7e70 2024-04-25T10:58:56.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.293+0000 7faf57c25640 1 -- 172.21.15.146:0/2402981647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf500b7e70 msgr2=0x7faf500d0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:56.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.293+0000 7faf57c25640 1 --2- 172.21.15.146:0/2402981647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf500b7e70 0x7faf500d0900 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7faf50072b80 tx=0x7faf4802a680 comp rx=0 tx=0).stop 2024-04-25T10:58:56.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.293+0000 7faf57c25640 1 -- 172.21.15.146:0/2402981647 shutdown_connections 2024-04-25T10:58:56.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.293+0000 7faf57c25640 1 --2- 172.21.15.146:0/2402981647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf500b7e70 0x7faf500d0900 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:56.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.293+0000 7faf57c25640 1 --2- 172.21.15.146:0/2402981647 >> v2:172.21.15.69:3300/0 conn(0x7faf500b1b20 0x7faf5006ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:56.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.293+0000 7faf57c25640 1 -- 172.21.15.146:0/2402981647 >> 172.21.15.146:0/2402981647 conn(0x7faf5009b7d0 msgr2=0x7faf5009bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:56.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.294+0000 7faf57c25640 1 -- 172.21.15.146:0/2402981647 shutdown_connections 2024-04-25T10:58:56.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.294+0000 7faf57c25640 1 -- 172.21.15.146:0/2402981647 wait complete. 2024-04-25T10:58:56.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.295+0000 7faf57c25640 1 Processor -- start 2024-04-25T10:58:56.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.295+0000 7faf57c25640 1 -- start start 2024-04-25T10:58:56.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.297+0000 7faf57c25640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf500b1b20 0x7faf500b0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:56.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.297+0000 7faf57c25640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf500c5250 0x7faf500b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:56.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.297+0000 7faf57c25640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf500b10b0 0x7faf500d0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:56.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.297+0000 7faf57c25640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7faf5015d6a0 con 0x7faf500b10b0 2024-04-25T10:58:56.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.297+0000 7faf57c25640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7faf5015d520 con 0x7faf500b1b20 2024-04-25T10:58:56.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.297+0000 7faf57c25640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7faf5006dc90 con 0x7faf500c5250 2024-04-25T10:58:56.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.297+0000 7faf559c1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf500b1b20 0x7faf500b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:56.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.297+0000 7faf559c1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf500b1b20 0x7faf500b0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57426/0 (socket says 172.21.15.146:57426) 2024-04-25T10:58:56.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.297+0000 7faf559c1640 1 -- 172.21.15.146:0/2836717121 learned_addr learned my addr 172.21.15.146:0/2836717121 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-04-25T10:58:56.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.297+0000 7faf551c0640 1 --2- 172.21.15.146:0/2836717121 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf500c5250 0x7faf500b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:56.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.297+0000 7faf561c2640 1 --2- 172.21.15.146:0/2836717121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf500b10b0 0x7faf500d0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:56.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.297+0000 7faf559c1640 1 -- 172.21.15.146:0/2836717121 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf500c5250 msgr2=0x7faf500b0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:56.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.297+0000 7faf559c1640 1 --2- 172.21.15.146:0/2836717121 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf500c5250 0x7faf500b0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:56.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.297+0000 7faf559c1640 1 -- 172.21.15.146:0/2836717121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf500b10b0 msgr2=0x7faf500d0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:56.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.297+0000 7faf559c1640 1 --2- 172.21.15.146:0/2836717121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf500b10b0 0x7faf500d0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:56.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.297+0000 7faf559c1640 1 -- 172.21.15.146:0/2836717121 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf500d0900 con 0x7faf500b1b20 2024-04-25T10:58:56.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.297+0000 7faf559c1640 1 --2- 172.21.15.146:0/2836717121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf500b1b20 0x7faf500b0630 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faf4400e870 tx=0x7faf4400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:56.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.297+0000 7faf427fc640 1 -- 172.21.15.146:0/2836717121 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf44017070 con 0x7faf500b1b20 2024-04-25T10:58:56.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.297+0000 7faf427fc640 1 -- 172.21.15.146:0/2836717121 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf4400c940 con 0x7faf500b1b20 2024-04-25T10:58:56.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.297+0000 7faf427fc640 1 -- 172.21.15.146:0/2836717121 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4400cd10 con 0x7faf500b1b20 2024-04-25T10:58:56.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.298+0000 7faf57c25640 1 -- 172.21.15.146:0/2836717121 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf500cbaf0 con 0x7faf500b1b20 2024-04-25T10:58:56.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.298+0000 7faf57c25640 1 -- 172.21.15.146:0/2836717121 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf500cc0b0 con 0x7faf500b1b20 2024-04-25T10:58:56.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.298+0000 7faf427fc640 1 -- 172.21.15.146:0/2836717121 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7faf4401a070 con 0x7faf500b1b20 2024-04-25T10:58:56.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.298+0000 7faf427fc640 1 -- 172.21.15.146:0/2836717121 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7faf4401ca40 con 0x7faf500b1b20 2024-04-25T10:58:56.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.298+0000 7faf57c25640 1 -- 172.21.15.146:0/2836717121 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf50050740 con 0x7faf500b1b20 2024-04-25T10:58:56.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.301+0000 7faf427fc640 1 -- 172.21.15.146:0/2836717121 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7faf4401c450 con 0x7faf500b1b20 2024-04-25T10:58:56.373 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.373+0000 7faf57c25640 1 -- 172.21.15.146:0/2836717121 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e0e6a832-b1a1-42d7-836d-b4906e9860a0", "id": 6} v 0) -- 0x7faf5009be90 con 0x7faf500b1b20 2024-04-25T10:58:56.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.379+0000 7faf427fc640 1 -- 172.21.15.146:0/2836717121 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e0e6a832-b1a1-42d7-836d-b4906e9860a0", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7faf4401c630 con 0x7faf500b1b20 2024-04-25T10:58:56.380 INFO:teuthology.orchestra.run.smithi146.stdout:6 2024-04-25T10:58:56.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.380+0000 7faf57c25640 1 -- 172.21.15.146:0/2836717121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf500b1b20 msgr2=0x7faf500b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:56.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.380+0000 7faf57c25640 1 --2- 172.21.15.146:0/2836717121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf500b1b20 0x7faf500b0630 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faf4400e870 tx=0x7faf4400ed30 comp rx=0 tx=0).stop 2024-04-25T10:58:56.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.380+0000 7faf57c25640 1 -- 172.21.15.146:0/2836717121 shutdown_connections 2024-04-25T10:58:56.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.380+0000 7faf57c25640 1 --2- 172.21.15.146:0/2836717121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf500b10b0 0x7faf500d0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:56.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.380+0000 7faf57c25640 1 --2- 172.21.15.146:0/2836717121 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf500c5250 0x7faf500b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:56.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.380+0000 7faf57c25640 1 --2- 172.21.15.146:0/2836717121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf500b1b20 0x7faf500b0630 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:56.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.380+0000 7faf57c25640 1 -- 172.21.15.146:0/2836717121 >> 172.21.15.146:0/2836717121 conn(0x7faf5009b7d0 msgr2=0x7faf50159a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:56.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.380+0000 7faf57c25640 1 -- 172.21.15.146:0/2836717121 shutdown_connections 2024-04-25T10:58:56.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.380+0000 7faf57c25640 1 -- 172.21.15.146:0/2836717121 wait complete. 2024-04-25T10:58:56.392 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new 25c077b9-5d8a-47ec-a83f-c8799112d7d7 7 2024-04-25T10:58:56.508 INFO:tasks.ceph.mgr.y.smithi069.stderr:2024-04-25T10:58:56.507+0000 7fecc21ee040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T10:58:56.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.540+0000 7fd061b7d640 1 Processor -- start 2024-04-25T10:58:56.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.540+0000 7fd061b7d640 1 -- start start 2024-04-25T10:58:56.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.541+0000 7fd061b7d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd05c0b5f80 0x7fd05c0ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:56.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.541+0000 7fd061b7d640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd05c0d2040 con 0x7fd05c0ce5a0 2024-04-25T10:58:56.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.541+0000 7fd061b7d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd05c0d3480 con 0x7fd05c0d3070 2024-04-25T10:58:56.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.541+0000 7fd061b7d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd05c0b6390 con 0x7fd05c0b5f80 2024-04-25T10:58:56.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.541+0000 7fd05affd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd05c0b5f80 0x7fd05c0ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:56.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.541+0000 7fd05b7fe640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fd05c0d3070 0x7fd05c06e740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:55444/0 (socket says 172.21.15.146:55444) 2024-04-25T10:58:56.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.541+0000 7fd05b7fe640 1 -- 172.21.15.146:0/4160493975 learned_addr learned my addr 172.21.15.146:0/4160493975 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-04-25T10:58:56.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.541+0000 7fd05a7fc640 1 -- 172.21.15.146:0/4160493975 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 343139799 0 0) 0x7fd05c0d3480 con 0x7fd05c0d3070 2024-04-25T10:58:56.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.541+0000 7fd05a7fc640 1 -- 172.21.15.146:0/4160493975 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd044003520 con 0x7fd05c0d3070 2024-04-25T10:58:56.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.542+0000 7fd05affd640 1 -- 172.21.15.146:0/4160493975 >> v1:172.21.15.146:6789/0 conn(0x7fd05c0d3070 legacy=0x7fd05c06e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:56.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.542+0000 7fd05affd640 1 -- 172.21.15.146:0/4160493975 >> v1:172.21.15.69:6789/0 conn(0x7fd05c0ce5a0 legacy=0x7fd05c160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:56.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.542+0000 7fd05affd640 1 -- 172.21.15.146:0/4160493975 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05c0d22b0 con 0x7fd05c0b5f80 2024-04-25T10:58:56.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.542+0000 7fd05affd640 1 --2- 172.21.15.146:0/4160493975 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd05c0b5f80 0x7fd05c0ce060 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd0500097d0 tx=0x7fd05002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=deb10b1f77e1ca85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:56.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.542+0000 7fd05a7fc640 1 -- 172.21.15.146:0/4160493975 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd050037070 con 0x7fd05c0b5f80 2024-04-25T10:58:56.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.542+0000 7fd05a7fc640 1 -- 172.21.15.146:0/4160493975 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd050032440 con 0x7fd05c0b5f80 2024-04-25T10:58:56.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.542+0000 7fd05a7fc640 1 -- 172.21.15.146:0/4160493975 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd050032810 con 0x7fd05c0b5f80 2024-04-25T10:58:56.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.543+0000 7fd061b7d640 1 -- 172.21.15.146:0/4160493975 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd05c0b5f80 msgr2=0x7fd05c0ce060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:56.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.543+0000 7fd061b7d640 1 --2- 172.21.15.146:0/4160493975 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd05c0b5f80 0x7fd05c0ce060 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd0500097d0 tx=0x7fd05002a210 comp rx=0 tx=0).stop 2024-04-25T10:58:56.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.543+0000 7fd061b7d640 1 -- 172.21.15.146:0/4160493975 shutdown_connections 2024-04-25T10:58:56.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.543+0000 7fd061b7d640 1 --2- 172.21.15.146:0/4160493975 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd05c0b5f80 0x7fd05c0ce060 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:56.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.543+0000 7fd061b7d640 1 -- 172.21.15.146:0/4160493975 >> 172.21.15.146:0/4160493975 conn(0x7fd05c0d3aa0 msgr2=0x7fd05c0d3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:56.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.543+0000 7fd061b7d640 1 -- 172.21.15.146:0/4160493975 shutdown_connections 2024-04-25T10:58:56.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.543+0000 7fd061b7d640 1 -- 172.21.15.146:0/4160493975 wait complete. 2024-04-25T10:58:56.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.545+0000 7fd061b7d640 1 Processor -- start 2024-04-25T10:58:56.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.545+0000 7fd061b7d640 1 -- start start 2024-04-25T10:58:56.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.546+0000 7fd061b7d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd05c0b5f80 0x7fd05c0cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:56.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.546+0000 7fd061b7d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd05c0ce5a0 0x7fd05c15a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:56.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.546+0000 7fd061b7d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd05c0d3070 0x7fd05c0b8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:56.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.546+0000 7fd061b7d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd05c0d2b50 con 0x7fd05c0ce5a0 2024-04-25T10:58:56.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.546+0000 7fd061b7d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd05c0d2870 con 0x7fd05c0b5f80 2024-04-25T10:58:56.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.546+0000 7fd061b7d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd05c0ce9b0 con 0x7fd05c0d3070 2024-04-25T10:58:56.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.547+0000 7fd05b7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd05c0b5f80 0x7fd05c0cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:56.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.547+0000 7fd05bfff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd05c0d3070 0x7fd05c0b8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:56.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.547+0000 7fd05b7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd05c0b5f80 0x7fd05c0cb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:57434/0 (socket says 172.21.15.146:57434) 2024-04-25T10:58:56.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.547+0000 7fd05bfff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd05c0d3070 0x7fd05c0b8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:44036/0 (socket says 172.21.15.146:44036) 2024-04-25T10:58:56.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.547+0000 7fd05b7fe640 1 -- 172.21.15.146:0/2536462231 learned_addr learned my addr 172.21.15.146:0/2536462231 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-04-25T10:58:56.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.547+0000 7fd05affd640 1 --2- 172.21.15.146:0/2536462231 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd05c0ce5a0 0x7fd05c15a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:56.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.547+0000 7fd05bfff640 1 -- 172.21.15.146:0/2536462231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd05c0b5f80 msgr2=0x7fd05c0cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:56.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.547+0000 7fd05bfff640 1 --2- 172.21.15.146:0/2536462231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd05c0b5f80 0x7fd05c0cb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:56.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.547+0000 7fd05bfff640 1 -- 172.21.15.146:0/2536462231 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd05c0ce5a0 msgr2=0x7fd05c15a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:56.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.547+0000 7fd05bfff640 1 --2- 172.21.15.146:0/2536462231 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd05c0ce5a0 0x7fd05c15a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:56.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.547+0000 7fd05bfff640 1 -- 172.21.15.146:0/2536462231 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05c0cd700 con 0x7fd05c0d3070 2024-04-25T10:58:56.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.547+0000 7fd05b7fe640 1 --2- 172.21.15.146:0/2536462231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd05c0b5f80 0x7fd05c0cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:58:56.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.547+0000 7fd05bfff640 1 --2- 172.21.15.146:0/2536462231 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd05c0d3070 0x7fd05c0b8070 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd05400c700 tx=0x7fd05400cbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:56.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.547+0000 7fd043fff640 1 -- 172.21.15.146:0/2536462231 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054003b70 con 0x7fd05c0d3070 2024-04-25T10:58:56.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.547+0000 7fd043fff640 1 -- 172.21.15.146:0/2536462231 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd054003d10 con 0x7fd05c0d3070 2024-04-25T10:58:56.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.547+0000 7fd061b7d640 1 -- 172.21.15.146:0/2536462231 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd05c0cc680 con 0x7fd05c0d3070 2024-04-25T10:58:56.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.547+0000 7fd043fff640 1 -- 172.21.15.146:0/2536462231 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054005310 con 0x7fd05c0d3070 2024-04-25T10:58:56.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.547+0000 7fd061b7d640 1 -- 172.21.15.146:0/2536462231 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd05c0cce60 con 0x7fd05c0d3070 2024-04-25T10:58:56.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.548+0000 7fd043fff640 1 -- 172.21.15.146:0/2536462231 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd0540054b0 con 0x7fd05c0d3070 2024-04-25T10:58:56.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.548+0000 7fd043fff640 1 -- 172.21.15.146:0/2536462231 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fd054012a80 con 0x7fd05c0d3070 2024-04-25T10:58:56.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.549+0000 7fd061b7d640 1 -- 172.21.15.146:0/2536462231 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd05c0cc8d0 con 0x7fd05c0d3070 2024-04-25T10:58:56.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.551+0000 7fd043fff640 1 -- 172.21.15.146:0/2536462231 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd054012d60 con 0x7fd05c0d3070 2024-04-25T10:58:56.607 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T10:58:56.606+0000 7f53b8f19040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T10:58:56.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.623+0000 7fd061b7d640 1 -- 172.21.15.146:0/2536462231 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd new", "uuid": "25c077b9-5d8a-47ec-a83f-c8799112d7d7", "id": 7} v 0) -- 0x7fd05c0d4920 con 0x7fd05c0d3070 2024-04-25T10:58:56.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.630+0000 7fd043fff640 1 -- 172.21.15.146:0/2536462231 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "25c077b9-5d8a-47ec-a83f-c8799112d7d7", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fd054012420 con 0x7fd05c0d3070 2024-04-25T10:58:56.630 INFO:teuthology.orchestra.run.smithi146.stdout:7 2024-04-25T10:58:56.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.630+0000 7fd061b7d640 1 -- 172.21.15.146:0/2536462231 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd05c0d3070 msgr2=0x7fd05c0b8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:56.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.631+0000 7fd061b7d640 1 --2- 172.21.15.146:0/2536462231 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd05c0d3070 0x7fd05c0b8070 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd05400c700 tx=0x7fd05400cbc0 comp rx=0 tx=0).stop 2024-04-25T10:58:56.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.631+0000 7fd061b7d640 1 -- 172.21.15.146:0/2536462231 shutdown_connections 2024-04-25T10:58:56.632 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.631+0000 7fd061b7d640 1 --2- 172.21.15.146:0/2536462231 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd05c0d3070 0x7fd05c0b8070 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:56.632 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.631+0000 7fd061b7d640 1 --2- 172.21.15.146:0/2536462231 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd05c0ce5a0 0x7fd05c15a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:56.632 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.631+0000 7fd061b7d640 1 --2- 172.21.15.146:0/2536462231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd05c0b5f80 0x7fd05c0cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:56.632 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.631+0000 7fd061b7d640 1 -- 172.21.15.146:0/2536462231 >> 172.21.15.146:0/2536462231 conn(0x7fd05c0d3aa0 msgr2=0x7fd05c15ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:56.632 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.631+0000 7fd061b7d640 1 -- 172.21.15.146:0/2536462231 shutdown_connections 2024-04-25T10:58:56.632 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:58:56.631+0000 7fd061b7d640 1 -- 172.21.15.146:0/2536462231 wait complete. 2024-04-25T10:58:56.642 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T10:58:56.642 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T10:58:56.645 INFO:tasks.ceph.osd.0:Started 2024-04-25T10:58:56.645 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T10:58:56.645 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T10:58:56.647 INFO:tasks.ceph.osd.1:Started 2024-04-25T10:58:56.647 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T10:58:56.647 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T10:58:56.650 INFO:tasks.ceph.osd.2:Started 2024-04-25T10:58:56.650 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T10:58:56.650 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T10:58:56.653 INFO:tasks.ceph.osd.3:Started 2024-04-25T10:58:56.653 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T10:58:56.653 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T10:58:56.655 INFO:tasks.ceph.osd.4:Started 2024-04-25T10:58:56.655 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T10:58:56.655 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T10:58:56.657 INFO:tasks.ceph.osd.5:Started 2024-04-25T10:58:56.657 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T10:58:56.658 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T10:58:56.660 INFO:tasks.ceph.osd.6:Started 2024-04-25T10:58:56.660 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T10:58:56.660 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T10:58:56.665 INFO:tasks.ceph.osd.7:Started 2024-04-25T10:58:56.666 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:58:56.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.807+0000 7ff77052e640 1 Processor -- start 2024-04-25T10:58:56.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.807+0000 7ff77052e640 1 -- start start 2024-04-25T10:58:56.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.808+0000 7ff77052e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7680c7230 0x7ff7680cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:56.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.809+0000 7ff77052e640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff76808e160 con 0x7ff7680b8f80 2024-04-25T10:58:56.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.809+0000 7ff77052e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff76808d8c0 con 0x7ff76808cfc0 2024-04-25T10:58:56.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.809+0000 7ff77052e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff7680b3480 con 0x7ff7680c7230 2024-04-25T10:58:56.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.809+0000 7ff76e2ca640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7ff7680b8f80 0x7ff7681649f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:53366/0 (socket says 172.21.15.69:53366) 2024-04-25T10:58:56.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.809+0000 7ff76e2ca640 1 -- 172.21.15.69:0/2620203266 learned_addr learned my addr 172.21.15.69:0/2620203266 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:58:56.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.809+0000 7ff76eacb640 1 --2- 172.21.15.69:0/2620203266 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7680c7230 0x7ff7680cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:56.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.809+0000 7ff76d2c8640 1 -- 172.21.15.69:0/2620203266 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2602768360 0 0) 0x7ff76808e160 con 0x7ff7680b8f80 2024-04-25T10:58:56.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.809+0000 7ff76d2c8640 1 -- 172.21.15.69:0/2620203266 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff754003520 con 0x7ff7680b8f80 2024-04-25T10:58:56.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.809+0000 7ff76d2c8640 1 -- 172.21.15.69:0/2620203266 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1980970959 0 0) 0x7ff76808d8c0 con 0x7ff76808cfc0 2024-04-25T10:58:56.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.809+0000 7ff76d2c8640 1 -- 172.21.15.69:0/2620203266 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff76808e160 con 0x7ff76808cfc0 2024-04-25T10:58:56.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.809+0000 7ff76eacb640 1 -- 172.21.15.69:0/2620203266 >> v1:172.21.15.146:6789/0 conn(0x7ff76808cfc0 legacy=0x7ff7680c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:56.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.809+0000 7ff76eacb640 1 -- 172.21.15.69:0/2620203266 >> v1:172.21.15.69:6789/0 conn(0x7ff7680b8f80 legacy=0x7ff7681649f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:56.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.809+0000 7ff76eacb640 1 -- 172.21.15.69:0/2620203266 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff76808dba0 con 0x7ff7680c7230 2024-04-25T10:58:56.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.810+0000 7ff76eacb640 1 --2- 172.21.15.69:0/2620203266 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7680c7230 0x7ff7680cf290 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff760009dc0 tx=0x7ff76002a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f119c1a1df8ba846 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:56.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.810+0000 7ff76d2c8640 1 -- 172.21.15.69:0/2620203266 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76002ae60 con 0x7ff7680c7230 2024-04-25T10:58:56.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.810+0000 7ff76d2c8640 1 -- 172.21.15.69:0/2620203266 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff760007740 con 0x7ff7680c7230 2024-04-25T10:58:56.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.810+0000 7ff76d2c8640 1 -- 172.21.15.69:0/2620203266 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff760007b10 con 0x7ff7680c7230 2024-04-25T10:58:56.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.811+0000 7ff77052e640 1 -- 172.21.15.69:0/2620203266 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7680c7230 msgr2=0x7ff7680cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:56.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.811+0000 7ff77052e640 1 --2- 172.21.15.69:0/2620203266 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7680c7230 0x7ff7680cf290 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff760009dc0 tx=0x7ff76002a640 comp rx=0 tx=0).stop 2024-04-25T10:58:56.813 INFO:tasks.ceph.mgr.y.smithi069.stderr:2024-04-25T10:58:56.811+0000 7fecc21ee040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:58:56.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.811+0000 7ff77052e640 1 -- 172.21.15.69:0/2620203266 shutdown_connections 2024-04-25T10:58:56.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.811+0000 7ff77052e640 1 --2- 172.21.15.69:0/2620203266 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7680c7230 0x7ff7680cf290 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:56.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.811+0000 7ff77052e640 1 -- 172.21.15.69:0/2620203266 >> 172.21.15.69:0/2620203266 conn(0x7ff7680b5640 msgr2=0x7ff7680b5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:56.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.811+0000 7ff77052e640 1 -- 172.21.15.69:0/2620203266 shutdown_connections 2024-04-25T10:58:56.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.811+0000 7ff77052e640 1 -- 172.21.15.69:0/2620203266 wait complete. 2024-04-25T10:58:56.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.813+0000 7ff77052e640 1 Processor -- start 2024-04-25T10:58:56.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.813+0000 7ff77052e640 1 -- start start 2024-04-25T10:58:56.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.814+0000 7ff77052e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff76808cfc0 0x7ff7680d0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:56.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.814+0000 7ff77052e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7680b8f80 0x7ff768157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:56.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.814+0000 7ff77052e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7680c7230 0x7ff7680b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:56.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.814+0000 7ff77052e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff7680b4aa0 con 0x7ff76808cfc0 2024-04-25T10:58:56.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.814+0000 7ff77052e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff7680b4920 con 0x7ff7680c7230 2024-04-25T10:58:56.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.814+0000 7ff77052e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff76808d3d0 con 0x7ff7680b8f80 2024-04-25T10:58:56.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.814+0000 7ff76e2ca640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff76808cfc0 0x7ff7680d0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:56.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.814+0000 7ff76e2ca640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff76808cfc0 0x7ff7680d0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35186/0 (socket says 172.21.15.69:35186) 2024-04-25T10:58:56.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.814+0000 7ff76e2ca640 1 -- 172.21.15.69:0/2485692584 learned_addr learned my addr 172.21.15.69:0/2485692584 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:58:56.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.815+0000 7ff76dac9640 1 --2- 172.21.15.69:0/2485692584 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7680b8f80 0x7ff768157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:56.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.815+0000 7ff76eacb640 1 --2- 172.21.15.69:0/2485692584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7680c7230 0x7ff7680b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:56.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.815+0000 7ff76e2ca640 1 -- 172.21.15.69:0/2485692584 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7680b8f80 msgr2=0x7ff768157fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:56.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.815+0000 7ff76e2ca640 1 --2- 172.21.15.69:0/2485692584 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7680b8f80 0x7ff768157fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:56.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.815+0000 7ff76e2ca640 1 -- 172.21.15.69:0/2485692584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7680c7230 msgr2=0x7ff7680b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:56.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.815+0000 7ff76e2ca640 1 --2- 172.21.15.69:0/2485692584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7680c7230 0x7ff7680b1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:56.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.815+0000 7ff76e2ca640 1 -- 172.21.15.69:0/2485692584 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7680cb370 con 0x7ff76808cfc0 2024-04-25T10:58:56.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.815+0000 7ff76e2ca640 1 --2- 172.21.15.69:0/2485692584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff76808cfc0 0x7ff7680d0520 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff758001260 tx=0x7ff75800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:56.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.815+0000 7ff75effd640 1 -- 172.21.15.69:0/2485692584 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff758017070 con 0x7ff76808cfc0 2024-04-25T10:58:56.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.815+0000 7ff75effd640 1 -- 172.21.15.69:0/2485692584 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7580074e0 con 0x7ff76808cfc0 2024-04-25T10:58:56.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.815+0000 7ff77052e640 1 -- 172.21.15.69:0/2485692584 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7680cb650 con 0x7ff76808cfc0 2024-04-25T10:58:56.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.815+0000 7ff75effd640 1 -- 172.21.15.69:0/2485692584 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7580059c0 con 0x7ff76808cfc0 2024-04-25T10:58:56.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.815+0000 7ff77052e640 1 -- 172.21.15.69:0/2485692584 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7680cbc10 con 0x7ff76808cfc0 2024-04-25T10:58:56.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.815+0000 7ff75effd640 1 -- 172.21.15.69:0/2485692584 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff75801a070 con 0x7ff76808cfc0 2024-04-25T10:58:56.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.816+0000 7ff75effd640 1 -- 172.21.15.69:0/2485692584 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7ff758005b60 con 0x7ff76808cfc0 2024-04-25T10:58:56.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.816+0000 7ff77052e640 1 -- 172.21.15.69:0/2485692584 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff768050740 con 0x7ff76808cfc0 2024-04-25T10:58:56.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.820+0000 7ff75effd640 1 -- 172.21.15.69:0/2485692584 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff758021020 con 0x7ff76808cfc0 2024-04-25T10:58:56.868 INFO:tasks.ceph.mgr.y.smithi069.stderr:2024-04-25T10:58:56.867+0000 7fecc21ee040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:58:56.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.882+0000 7ff77052e640 1 -- 172.21.15.69:0/2485692584 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff7680b30e0 con 0x7ff76808cfc0 2024-04-25T10:58:56.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.883+0000 7ff75effd640 1 -- 172.21.15.69:0/2485692584 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7ff75801c440 con 0x7ff76808cfc0 2024-04-25T10:58:56.884 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:58:56.884 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":10,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T10:58:56.625383+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:58:56.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.883+0000 7ff77052e640 1 -- 172.21.15.69:0/2485692584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff76808cfc0 msgr2=0x7ff7680d0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:56.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.883+0000 7ff77052e640 1 --2- 172.21.15.69:0/2485692584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff76808cfc0 0x7ff7680d0520 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff758001260 tx=0x7ff75800c890 comp rx=0 tx=0).stop 2024-04-25T10:58:56.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.883+0000 7ff77052e640 1 -- 172.21.15.69:0/2485692584 shutdown_connections 2024-04-25T10:58:56.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.883+0000 7ff77052e640 1 --2- 172.21.15.69:0/2485692584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7680c7230 0x7ff7680b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:56.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.883+0000 7ff77052e640 1 --2- 172.21.15.69:0/2485692584 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7680b8f80 0x7ff768157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:56.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.883+0000 7ff77052e640 1 --2- 172.21.15.69:0/2485692584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff76808cfc0 0x7ff7680d0520 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:56.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.884+0000 7ff77052e640 1 -- 172.21.15.69:0/2485692584 >> 172.21.15.69:0/2485692584 conn(0x7ff7680b5640 msgr2=0x7ff7680d29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:56.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.884+0000 7ff77052e640 1 -- 172.21.15.69:0/2485692584 shutdown_connections 2024-04-25T10:58:56.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:58:56.884+0000 7ff77052e640 1 -- 172.21.15.69:0/2485692584 wait complete. 2024-04-25T10:58:56.895 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T10:58:56.895 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T10:58:56.912 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T10:58:56.911+0000 7f53b8f19040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:58:56.964 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T10:58:56.964+0000 7f53b8f19040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:58:56.987 INFO:tasks.ceph.mgr.y.smithi069.stderr:2024-04-25T10:58:56.986+0000 7fecc21ee040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T10:58:57.082 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T10:58:57.082+0000 7f53b8f19040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T10:58:57.264 INFO:tasks.ceph.osd.1.smithi069.stderr:2024-04-25T10:58:57.263+0000 7f2eda615640 -1 Falling back to public interface 2024-04-25T10:58:57.267 INFO:tasks.ceph.mgr.y.smithi069.stderr:2024-04-25T10:58:57.266+0000 7fecc21ee040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:58:57.272 INFO:tasks.ceph.osd.4.smithi146.stderr:2024-04-25T10:58:57.271+0000 7f107fa19640 -1 Falling back to public interface 2024-04-25T10:58:57.278 INFO:tasks.ceph.osd.3.smithi069.stderr:2024-04-25T10:58:57.277+0000 7f9de1020640 -1 Falling back to public interface 2024-04-25T10:58:57.280 INFO:tasks.ceph.osd.5.smithi146.stderr:2024-04-25T10:58:57.279+0000 7ff86ec0f640 -1 Falling back to public interface 2024-04-25T10:58:57.280 INFO:tasks.ceph.osd.6.smithi146.stderr:2024-04-25T10:58:57.279+0000 7f0623c1e640 -1 Falling back to public interface 2024-04-25T10:58:57.281 INFO:tasks.ceph.osd.2.smithi069.stderr:2024-04-25T10:58:57.280+0000 7f97abc0d640 -1 Falling back to public interface 2024-04-25T10:58:57.286 INFO:tasks.ceph.osd.0.smithi069.stderr:2024-04-25T10:58:57.286+0000 7f1548edf640 -1 Falling back to public interface 2024-04-25T10:58:57.303 INFO:tasks.ceph.osd.7.smithi146.stderr:2024-04-25T10:58:57.303+0000 7f27065b1640 -1 Falling back to public interface 2024-04-25T10:58:57.321 INFO:tasks.ceph.mgr.y.smithi069.stderr:2024-04-25T10:58:57.321+0000 7fecc21ee040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:58:57.369 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T10:58:57.369+0000 7f53b8f19040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:58:57.417 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T10:58:57.416+0000 7f53b8f19040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:58:57.476 INFO:tasks.ceph.mgr.y.smithi069.stderr:2024-04-25T10:58:57.476+0000 7fecc21ee040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:58:57.576 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T10:58:57.575+0000 7f53b8f19040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:58:57.614 INFO:tasks.ceph.mgr.y.smithi069.stderr:2024-04-25T10:58:57.614+0000 7fecc21ee040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:58:57.664 INFO:tasks.ceph.mgr.y.smithi069.stderr:2024-04-25T10:58:57.663+0000 7fecc21ee040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:58:57.700 DEBUG:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:58:57.712 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T10:58:57.712+0000 7f53b8f19040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:58:57.735 INFO:tasks.ceph.mgr.y.smithi069.stderr:2024-04-25T10:58:57.735+0000 7fecc21ee040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:58:57.761 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T10:58:57.761+0000 7f53b8f19040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:58:57.784 INFO:tasks.ceph.mgr.y.smithi069.stderr:2024-04-25T10:58:57.783+0000 7fecc21ee040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:58:57.801 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.801+0000 7f48dd251640 1 Processor -- start 2024-04-25T10:58:57.801 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.801+0000 7f48dd251640 1 -- start start 2024-04-25T10:58:57.803 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.802+0000 7f48dd251640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f48d806f6d0 0x7f48d80d29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:57.803 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.802+0000 7f48dd251640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48d80d3330 con 0x7f48d806f2c0 2024-04-25T10:58:57.803 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.802+0000 7f48dd251640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48d80b3230 con 0x7f48d80d2f20 2024-04-25T10:58:57.803 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.802+0000 7f48dd251640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f48d80b43a0 con 0x7f48d806f6d0 2024-04-25T10:58:57.803 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.802+0000 7f48cffff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f48d806f6d0 0x7f48d80d29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:57.803 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.802+0000 7f48d6d9d640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f48d806f2c0 0x7f48d80b3c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:53378/0 (socket says 172.21.15.69:53378) 2024-04-25T10:58:57.803 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.802+0000 7f48d6d9d640 1 -- 172.21.15.69:0/1923556314 learned_addr learned my addr 172.21.15.69:0/1923556314 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:58:57.804 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.803+0000 7f48cffff640 1 -- 172.21.15.69:0/1923556314 >> v1:172.21.15.146:6789/0 conn(0x7f48d80d2f20 legacy=0x7f48d80cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:57.804 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.803+0000 7f48cffff640 1 -- 172.21.15.69:0/1923556314 >> v1:172.21.15.69:6789/0 conn(0x7f48d806f2c0 legacy=0x7f48d80b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:57.804 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.803+0000 7f48d659c640 1 -- 172.21.15.69:0/1923556314 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1254404989 0 0) 0x7f48c8001110 con 0x7f48d806f2c0 2024-04-25T10:58:57.804 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.803+0000 7f48cffff640 1 -- 172.21.15.69:0/1923556314 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d80b3510 con 0x7f48d806f6d0 2024-04-25T10:58:57.805 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.803+0000 7f48cffff640 1 --2- 172.21.15.69:0/1923556314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48d806f6d0 0x7f48d80d29e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f48c00097a0 tx=0x7f48c002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37a5cbd7ed68c91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:57.805 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.803+0000 7f48d659c640 1 -- 172.21.15.69:0/1923556314 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c0037070 con 0x7f48d806f6d0 2024-04-25T10:58:57.805 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.803+0000 7f48d659c640 1 -- 172.21.15.69:0/1923556314 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48c0032440 con 0x7f48d806f6d0 2024-04-25T10:58:57.805 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.803+0000 7f48d659c640 1 -- 172.21.15.69:0/1923556314 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c0032810 con 0x7f48d806f6d0 2024-04-25T10:58:57.805 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.804+0000 7f48dd251640 1 -- 172.21.15.69:0/1923556314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48d806f6d0 msgr2=0x7f48d80d29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:57.805 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.804+0000 7f48dd251640 1 --2- 172.21.15.69:0/1923556314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48d806f6d0 0x7f48d80d29e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f48c00097a0 tx=0x7f48c002a230 comp rx=0 tx=0).stop 2024-04-25T10:58:57.805 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.804+0000 7f48dd251640 1 -- 172.21.15.69:0/1923556314 shutdown_connections 2024-04-25T10:58:57.806 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.804+0000 7f48dd251640 1 --2- 172.21.15.69:0/1923556314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48d806f6d0 0x7f48d80d29e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:57.806 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.804+0000 7f48dd251640 1 -- 172.21.15.69:0/1923556314 >> 172.21.15.69:0/1923556314 conn(0x7f48d809beb0 msgr2=0x7f48d80b59c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:57.806 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.804+0000 7f48dd251640 1 -- 172.21.15.69:0/1923556314 shutdown_connections 2024-04-25T10:58:57.806 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.804+0000 7f48dd251640 1 -- 172.21.15.69:0/1923556314 wait complete. 2024-04-25T10:58:57.807 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.806+0000 7f48dd251640 1 Processor -- start 2024-04-25T10:58:57.807 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.806+0000 7f48dd251640 1 -- start start 2024-04-25T10:58:57.810 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.807+0000 7f48dd251640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48d806f2c0 0x7f48d814d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:57.810 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.807+0000 7f48dd251640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48d806f6d0 0x7f48d809e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:57.810 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.808+0000 7f48dd251640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48d80d2f20 0x7f48d806d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:58:57.810 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.808+0000 7f48dd251640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f48d80ce940 con 0x7f48d806f6d0 2024-04-25T10:58:57.810 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.808+0000 7f48dd251640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f48d80ce660 con 0x7f48d806f2c0 2024-04-25T10:58:57.810 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.808+0000 7f48dd251640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f48d80b2db0 con 0x7f48d80d2f20 2024-04-25T10:58:57.810 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.808+0000 7f48cffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48d806f6d0 0x7f48d809e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:57.810 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.808+0000 7f48cffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48d806f6d0 0x7f48d809e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35202/0 (socket says 172.21.15.69:35202) 2024-04-25T10:58:57.810 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.808+0000 7f48cffff640 1 -- 172.21.15.69:0/3623115159 learned_addr learned my addr 172.21.15.69:0/3623115159 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:58:57.811 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.808+0000 7f48d6d9d640 1 --2- 172.21.15.69:0/3623115159 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48d806f2c0 0x7f48d814d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:57.811 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.808+0000 7f48d759e640 1 --2- 172.21.15.69:0/3623115159 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48d80d2f20 0x7f48d806d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:58:57.811 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.808+0000 7f48cffff640 1 -- 172.21.15.69:0/3623115159 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48d80d2f20 msgr2=0x7f48d806d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:57.811 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.808+0000 7f48cffff640 1 --2- 172.21.15.69:0/3623115159 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48d80d2f20 0x7f48d806d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:57.811 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.808+0000 7f48cffff640 1 -- 172.21.15.69:0/3623115159 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48d806f2c0 msgr2=0x7f48d814d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:57.811 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.808+0000 7f48cffff640 1 --2- 172.21.15.69:0/3623115159 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48d806f2c0 0x7f48d814d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:57.811 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.808+0000 7f48cffff640 1 -- 172.21.15.69:0/3623115159 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d814d600 con 0x7f48d806f6d0 2024-04-25T10:58:57.811 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.808+0000 7f48cffff640 1 --2- 172.21.15.69:0/3623115159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48d806f6d0 0x7f48d809e8a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f48c00098c0 tx=0x7f48c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:58:57.811 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.808+0000 7f48cf7fe640 1 -- 172.21.15.69:0/3623115159 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c0037040 con 0x7f48d806f6d0 2024-04-25T10:58:57.811 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.808+0000 7f48cf7fe640 1 -- 172.21.15.69:0/3623115159 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48c0042070 con 0x7f48d806f6d0 2024-04-25T10:58:57.811 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.808+0000 7f48dd251640 1 -- 172.21.15.69:0/3623115159 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48d809f5d0 con 0x7f48d806f6d0 2024-04-25T10:58:57.811 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.808+0000 7f48cf7fe640 1 -- 172.21.15.69:0/3623115159 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c0030a70 con 0x7f48d806f6d0 2024-04-25T10:58:57.811 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.808+0000 7f48dd251640 1 -- 172.21.15.69:0/3623115159 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48d809fb90 con 0x7f48d806f6d0 2024-04-25T10:58:57.811 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.809+0000 7f48cf7fe640 1 -- 172.21.15.69:0/3623115159 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f48c0030c10 con 0x7f48d806f6d0 2024-04-25T10:58:57.812 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.809+0000 7f48cf7fe640 1 -- 172.21.15.69:0/3623115159 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f48c0047d70 con 0x7f48d806f6d0 2024-04-25T10:58:57.812 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.810+0000 7f48dd251640 1 -- 172.21.15.69:0/3623115159 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48d809f2f0 con 0x7f48d806f6d0 2024-04-25T10:58:57.812 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.812+0000 7f48cf7fe640 1 -- 172.21.15.69:0/3623115159 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f48d814d600 con 0x7f48d806f6d0 2024-04-25T10:58:57.832 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T10:58:57.832+0000 7f53b8f19040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:58:57.874 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.874+0000 7f48dd251640 1 -- 172.21.15.69:0/3623115159 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f48d806cef0 con 0x7f48d806f6d0 2024-04-25T10:58:57.875 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.874+0000 7f48cf7fe640 1 -- 172.21.15.69:0/3623115159 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f48c003db60 con 0x7f48d806f6d0 2024-04-25T10:58:57.875 INFO:teuthology.misc.health.smithi069.stdout: 2024-04-25T10:58:57.876 INFO:teuthology.misc.health.smithi069.stdout:{"epoch":10,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T10:58:56.625383+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:58:57.877 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.876+0000 7f48dd251640 1 -- 172.21.15.69:0/3623115159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48d806f6d0 msgr2=0x7f48d809e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:58:57.877 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.876+0000 7f48dd251640 1 --2- 172.21.15.69:0/3623115159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48d806f6d0 0x7f48d809e8a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f48c00098c0 tx=0x7f48c00046c0 comp rx=0 tx=0).stop 2024-04-25T10:58:57.877 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.876+0000 7f48dd251640 1 -- 172.21.15.69:0/3623115159 shutdown_connections 2024-04-25T10:58:57.877 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.876+0000 7f48dd251640 1 --2- 172.21.15.69:0/3623115159 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48d80d2f20 0x7f48d806d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:57.877 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.876+0000 7f48dd251640 1 --2- 172.21.15.69:0/3623115159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48d806f6d0 0x7f48d809e8a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:57.877 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.876+0000 7f48dd251640 1 --2- 172.21.15.69:0/3623115159 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48d806f2c0 0x7f48d814d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:58:57.877 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.876+0000 7f48dd251640 1 -- 172.21.15.69:0/3623115159 >> 172.21.15.69:0/3623115159 conn(0x7f48d809beb0 msgr2=0x7f48d8071010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:58:57.877 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.876+0000 7f48dd251640 1 -- 172.21.15.69:0/3623115159 shutdown_connections 2024-04-25T10:58:57.877 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:58:57.876+0000 7f48dd251640 1 -- 172.21.15.69:0/3623115159 wait complete. 2024-04-25T10:58:57.879 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T10:58:57.879+0000 7f53b8f19040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:58:57.890 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T10:58:58.273 INFO:tasks.ceph.mgr.y.smithi069.stderr:2024-04-25T10:58:58.273+0000 7fecc21ee040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:58:58.358 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T10:58:58.357+0000 7f53b8f19040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:58:58.392 INFO:tasks.ceph.mgr.y.smithi069.stderr:2024-04-25T10:58:58.391+0000 7fecc21ee040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:58:58.481 INFO:tasks.ceph.mgr.x.smithi146.stderr:2024-04-25T10:58:58.480+0000 7f53b8f19040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:58:59.409 INFO:tasks.ceph.mgr.y.smithi069.stderr:2024-04-25T10:58:59.409+0000 7fec62c7a640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T10:58:59.410 INFO:tasks.ceph.mgr.y.smithi069.stderr:2024-04-25T10:58:59.409+0000 7fec62c7a640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T10:59:00.778 INFO:tasks.ceph.osd.2.smithi069.stderr:2024-04-25T10:59:00.777+0000 7f97abc0d640 -1 osd.2 0 log_to_monitors true 2024-04-25T10:59:00.788 INFO:tasks.ceph.osd.4.smithi146.stderr:2024-04-25T10:59:00.787+0000 7f107fa19640 -1 osd.4 0 log_to_monitors true 2024-04-25T10:59:00.801 INFO:tasks.ceph.osd.1.smithi069.stderr:2024-04-25T10:59:00.800+0000 7f2eda615640 -1 osd.1 0 log_to_monitors true 2024-04-25T10:59:00.820 INFO:tasks.ceph.osd.5.smithi146.stderr:2024-04-25T10:59:00.819+0000 7ff86ec0f640 -1 osd.5 0 log_to_monitors true 2024-04-25T10:59:00.820 INFO:tasks.ceph.osd.6.smithi146.stderr:2024-04-25T10:59:00.820+0000 7f0623c1e640 -1 osd.6 0 log_to_monitors true 2024-04-25T10:59:00.820 INFO:tasks.ceph.osd.3.smithi069.stderr:2024-04-25T10:59:00.820+0000 7f9de1020640 -1 osd.3 0 log_to_monitors true 2024-04-25T10:59:00.821 INFO:tasks.ceph.osd.7.smithi146.stderr:2024-04-25T10:59:00.820+0000 7f27065b1640 -1 osd.7 0 log_to_monitors true 2024-04-25T10:59:00.822 INFO:tasks.ceph.osd.0.smithi069.stderr:2024-04-25T10:59:00.822+0000 7f1548edf640 -1 osd.0 0 log_to_monitors true 2024-04-25T10:59:04.519 INFO:tasks.ceph.osd.6.smithi146.stderr:2024-04-25T10:59:04.519+0000 7f061f50c640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T10:59:04.536 INFO:tasks.ceph.osd.3.smithi069.stderr:2024-04-25T10:59:04.536+0000 7f9ddc917640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T10:59:04.542 INFO:tasks.ceph.osd.1.smithi069.stderr:2024-04-25T10:59:04.542+0000 7f2ed5f2c640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T10:59:04.543 INFO:tasks.ceph.osd.6.smithi146.stderr:2024-04-25T10:59:04.543+0000 7f0617318640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:59:04.546 INFO:tasks.ceph.osd.3.smithi069.stderr:2024-04-25T10:59:04.546+0000 7f9dd4712640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:59:04.549 INFO:tasks.ceph.osd.4.smithi146.stderr:2024-04-25T10:59:04.549+0000 7f107b9af640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T10:59:04.555 INFO:tasks.ceph.osd.0.smithi069.stderr:2024-04-25T10:59:04.554+0000 7f1544e75640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T10:59:04.556 INFO:tasks.ceph.osd.1.smithi069.stderr:2024-04-25T10:59:04.556+0000 7f2ecdd27640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:59:04.560 INFO:tasks.ceph.osd.5.smithi146.stderr:2024-04-25T10:59:04.559+0000 7ff86a622640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T10:59:04.560 INFO:tasks.ceph.osd.2.smithi069.stderr:2024-04-25T10:59:04.560+0000 7f97a7598640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T10:59:04.563 INFO:tasks.ceph.osd.4.smithi146.stderr:2024-04-25T10:59:04.563+0000 7f10737aa640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:59:04.568 INFO:tasks.ceph.osd.0.smithi069.stderr:2024-04-25T10:59:04.568+0000 7f153cc70640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:59:04.573 INFO:tasks.ceph.osd.2.smithi069.stderr:2024-04-25T10:59:04.573+0000 7f979f393640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:59:04.579 INFO:tasks.ceph.osd.5.smithi146.stderr:2024-04-25T10:59:04.579+0000 7ff86241d640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:59:04.639 INFO:tasks.ceph.osd.7.smithi146.stderr:2024-04-25T10:59:04.639+0000 7f2702547640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T10:59:04.650 INFO:tasks.ceph.osd.7.smithi146.stderr:2024-04-25T10:59:04.650+0000 7f26fa342640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:59:04.697 DEBUG:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:59:04.797 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.796+0000 7f1bd4e9a640 1 Processor -- start 2024-04-25T10:59:04.797 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.796+0000 7f1bd4e9a640 1 -- start start 2024-04-25T10:59:04.799 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.798+0000 7f1bd4e9a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1bd00ce7a0 0x7f1bd01604f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:04.799 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.798+0000 7f1bd4e9a640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1bd0164af0 0x7f1bd006e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:04.799 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.798+0000 7f1bd4e9a640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bd0164f00 con 0x7f1bd00b5680 2024-04-25T10:59:04.799 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.798+0000 7f1bd4e9a640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f1bd00d17d0 con 0x7f1bd0164af0 2024-04-25T10:59:04.800 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.798+0000 7f1bd4e9a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1bd006e9d0 con 0x7f1bd00ce7a0 2024-04-25T10:59:04.800 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.798+0000 7f1bce59c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1bd00ce7a0 0x7f1bd01604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:04.800 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.798+0000 7f1bcdd9b640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1bd0164af0 0x7f1bd006e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:04.800 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.798+0000 7f1bce59c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1bd00ce7a0 0x7f1bd01604f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.69:51508/0 (socket says 172.21.15.69:51508) 2024-04-25T10:59:04.800 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.798+0000 7f1bce59c640 1 -- 172.21.15.69:0/3813262865 learned_addr learned my addr 172.21.15.69:0/3813262865 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:04.800 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.798+0000 7f1bcdd9b640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1bd0164af0 0x7f1bd006e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.69:54708/0 (socket says 172.21.15.69:54708) 2024-04-25T10:59:04.800 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.799+0000 7f1bce59c640 1 -- 172.21.15.69:0/3813262865 >> v2:172.21.15.146:3300/0 conn(0x7f1bd0164af0 msgr2=0x7f1bd006e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:04.800 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.799+0000 7f1bce59c640 1 --2- 172.21.15.69:0/3813262865 >> v2:172.21.15.146:3300/0 conn(0x7f1bd0164af0 0x7f1bd006e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:04.800 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.799+0000 7f1bce59c640 1 -- 172.21.15.69:0/3813262865 >> v1:172.21.15.146:6789/0 conn(0x7f1bd00b5680 legacy=0x7f1bd00b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:04.800 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.799+0000 7f1bce59c640 1 -- 172.21.15.69:0/3813262865 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd006ec70 con 0x7f1bd00ce7a0 2024-04-25T10:59:04.800 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.799+0000 7f1bcdd9b640 1 --2- 172.21.15.69:0/3813262865 >> v2:172.21.15.146:3300/0 conn(0x7f1bd0164af0 0x7f1bd006e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:59:04.802 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.799+0000 7f1bce59c640 1 --2- 172.21.15.69:0/3813262865 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1bd00ce7a0 0x7f1bd01604f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8009870 tx=0x7f1bc802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32d763c0ea4f503d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:04.802 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.799+0000 7f1bcd59a640 1 -- 172.21.15.69:0/3813262865 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8037070 con 0x7f1bd00ce7a0 2024-04-25T10:59:04.802 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.799+0000 7f1bcd59a640 1 -- 172.21.15.69:0/3813262865 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bc8032440 con 0x7f1bd00ce7a0 2024-04-25T10:59:04.802 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.799+0000 7f1bcd59a640 1 -- 172.21.15.69:0/3813262865 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8032810 con 0x7f1bd00ce7a0 2024-04-25T10:59:04.802 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.800+0000 7f1bd4e9a640 1 -- 172.21.15.69:0/3813262865 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1bd00ce7a0 msgr2=0x7f1bd01604f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:04.802 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.800+0000 7f1bd4e9a640 1 --2- 172.21.15.69:0/3813262865 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1bd00ce7a0 0x7f1bd01604f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8009870 tx=0x7f1bc802a1c0 comp rx=0 tx=0).stop 2024-04-25T10:59:04.803 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.800+0000 7f1bd4e9a640 1 -- 172.21.15.69:0/3813262865 shutdown_connections 2024-04-25T10:59:04.803 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.800+0000 7f1bd4e9a640 1 --2- 172.21.15.69:0/3813262865 >> v2:172.21.15.146:3300/0 conn(0x7f1bd0164af0 0x7f1bd006e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:04.803 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.800+0000 7f1bd4e9a640 1 --2- 172.21.15.69:0/3813262865 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1bd00ce7a0 0x7f1bd01604f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:04.803 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.800+0000 7f1bd4e9a640 1 -- 172.21.15.69:0/3813262865 >> 172.21.15.69:0/3813262865 conn(0x7f1bd00d0680 msgr2=0x7f1bd00d0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:04.803 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.800+0000 7f1bd4e9a640 1 -- 172.21.15.69:0/3813262865 shutdown_connections 2024-04-25T10:59:04.803 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.800+0000 7f1bd4e9a640 1 -- 172.21.15.69:0/3813262865 wait complete. 2024-04-25T10:59:04.803 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.802+0000 7f1bd4e9a640 1 Processor -- start 2024-04-25T10:59:04.804 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.802+0000 7f1bd4e9a640 1 -- start start 2024-04-25T10:59:04.806 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.803+0000 7f1bd4e9a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1bd00b5680 0x7f1bd0150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:04.806 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.803+0000 7f1bd4e9a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1bd00ce7a0 0x7f1bd01f5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:04.806 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.803+0000 7f1bd4e9a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1bd0164af0 0x7f1bd01f63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:04.806 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.804+0000 7f1bd4e9a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd006f510 con 0x7f1bd00ce7a0 2024-04-25T10:59:04.806 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.804+0000 7f1bd4e9a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd006f230 con 0x7f1bd0164af0 2024-04-25T10:59:04.806 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.804+0000 7f1bd4e9a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1bd00d1c60 con 0x7f1bd00b5680 2024-04-25T10:59:04.806 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.804+0000 7f1bcdd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1bd00ce7a0 0x7f1bd01f5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:04.806 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.804+0000 7f1bcdd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1bd00ce7a0 0x7f1bd01f5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35332/0 (socket says 172.21.15.69:35332) 2024-04-25T10:59:04.806 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.804+0000 7f1bcdd9b640 1 -- 172.21.15.69:0/2039352758 learned_addr learned my addr 172.21.15.69:0/2039352758 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:04.807 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.804+0000 7f1bce59c640 1 --2- 172.21.15.69:0/2039352758 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1bd00b5680 0x7f1bd0150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:04.807 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.804+0000 7f1bced9d640 1 --2- 172.21.15.69:0/2039352758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1bd0164af0 0x7f1bd01f63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:04.807 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.804+0000 7f1bcdd9b640 1 -- 172.21.15.69:0/2039352758 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1bd00b5680 msgr2=0x7f1bd0150480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:04.807 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.804+0000 7f1bcdd9b640 1 --2- 172.21.15.69:0/2039352758 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1bd00b5680 0x7f1bd0150480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:04.807 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.804+0000 7f1bcdd9b640 1 -- 172.21.15.69:0/2039352758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1bd0164af0 msgr2=0x7f1bd01f63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:04.807 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.804+0000 7f1bcdd9b640 1 --2- 172.21.15.69:0/2039352758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1bd0164af0 0x7f1bd01f63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:04.807 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.804+0000 7f1bcdd9b640 1 -- 172.21.15.69:0/2039352758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd0161680 con 0x7f1bd00ce7a0 2024-04-25T10:59:04.807 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.804+0000 7f1bcdd9b640 1 --2- 172.21.15.69:0/2039352758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1bd00ce7a0 0x7f1bd01f5eb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4009e00 tx=0x7f1bc400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:04.807 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.804+0000 7f1bbeffd640 1 -- 172.21.15.69:0/2039352758 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4018020 con 0x7f1bd00ce7a0 2024-04-25T10:59:04.807 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.804+0000 7f1bbeffd640 1 -- 172.21.15.69:0/2039352758 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bc4014680 con 0x7f1bd00ce7a0 2024-04-25T10:59:04.807 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.804+0000 7f1bd4e9a640 1 -- 172.21.15.69:0/2039352758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bd0161960 con 0x7f1bd00ce7a0 2024-04-25T10:59:04.807 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.804+0000 7f1bbeffd640 1 -- 172.21.15.69:0/2039352758 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4014a70 con 0x7f1bd00ce7a0 2024-04-25T10:59:04.807 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.804+0000 7f1bd4e9a640 1 -- 172.21.15.69:0/2039352758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bd0161f20 con 0x7f1bd00ce7a0 2024-04-25T10:59:04.807 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.805+0000 7f1bd4e9a640 1 -- 172.21.15.69:0/2039352758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bd006a160 con 0x7f1bd00ce7a0 2024-04-25T10:59:04.808 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.806+0000 7f1bbeffd640 1 -- 172.21.15.69:0/2039352758 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 4) ==== 96726+0+0 (secure 0 0 0) 0x7f1bc4005d10 con 0x7f1bd00ce7a0 2024-04-25T10:59:04.808 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.806+0000 7f1bbeffd640 1 --2- 172.21.15.69:0/2039352758 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1bb0074420 0x7f1bb0076930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:04.808 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.806+0000 7f1bbeffd640 1 -- 172.21.15.69:0/2039352758 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f1bc4095010 con 0x7f1bd00ce7a0 2024-04-25T10:59:04.808 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.806+0000 7f1bce59c640 1 --2- 172.21.15.69:0/2039352758 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1bb0074420 0x7f1bb0076930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:04.809 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.807+0000 7f1bce59c640 1 --2- 172.21.15.69:0/2039352758 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1bb0074420 0x7f1bb0076930 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8034000 tx=0x7f1bc802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:04.812 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.811+0000 7f1bbeffd640 1 -- 172.21.15.69:0/2039352758 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1bc4022030 con 0x7f1bd00ce7a0 2024-04-25T10:59:04.995 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.994+0000 7f1bd4e9a640 1 -- 172.21.15.69:0/2039352758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1bd006dd20 con 0x7f1bd00ce7a0 2024-04-25T10:59:04.995 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.995+0000 7f1bbeffd640 1 -- 172.21.15.69:0/2039352758 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f1bc4010040 con 0x7f1bd00ce7a0 2024-04-25T10:59:04.995 INFO:teuthology.misc.health.smithi069.stdout: 2024-04-25T10:59:04.996 INFO:teuthology.misc.health.smithi069.stdout:{"epoch":12,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T10:59:02.410049+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:59:04.998 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.997+0000 7f1bd4e9a640 1 -- 172.21.15.69:0/2039352758 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1bb0074420 msgr2=0x7f1bb0076930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:04.998 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.997+0000 7f1bd4e9a640 1 --2- 172.21.15.69:0/2039352758 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1bb0074420 0x7f1bb0076930 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8034000 tx=0x7f1bc802e000 comp rx=0 tx=0).stop 2024-04-25T10:59:04.998 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.997+0000 7f1bd4e9a640 1 -- 172.21.15.69:0/2039352758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1bd00ce7a0 msgr2=0x7f1bd01f5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:04.998 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.997+0000 7f1bd4e9a640 1 --2- 172.21.15.69:0/2039352758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1bd00ce7a0 0x7f1bd01f5eb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4009e00 tx=0x7f1bc400c490 comp rx=0 tx=0).stop 2024-04-25T10:59:04.998 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.997+0000 7f1bd4e9a640 1 -- 172.21.15.69:0/2039352758 shutdown_connections 2024-04-25T10:59:04.998 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.997+0000 7f1bd4e9a640 1 --2- 172.21.15.69:0/2039352758 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1bb0074420 0x7f1bb0076930 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:04.998 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.997+0000 7f1bd4e9a640 1 --2- 172.21.15.69:0/2039352758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1bd0164af0 0x7f1bd01f63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:04.998 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.997+0000 7f1bd4e9a640 1 --2- 172.21.15.69:0/2039352758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1bd00ce7a0 0x7f1bd01f5eb0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:04.998 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.997+0000 7f1bd4e9a640 1 --2- 172.21.15.69:0/2039352758 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1bd00b5680 0x7f1bd0150480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:04.998 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.997+0000 7f1bd4e9a640 1 -- 172.21.15.69:0/2039352758 >> 172.21.15.69:0/2039352758 conn(0x7f1bd00d0680 msgr2=0x7f1bd00b5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:04.998 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.997+0000 7f1bd4e9a640 1 -- 172.21.15.69:0/2039352758 shutdown_connections 2024-04-25T10:59:04.998 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:04.997+0000 7f1bd4e9a640 1 -- 172.21.15.69:0/2039352758 wait complete. 2024-04-25T10:59:05.007 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T10:59:11.815 DEBUG:teuthology.orchestra.run.smithi069:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:59:11.928 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.928+0000 7f27f37d6640 1 Processor -- start 2024-04-25T10:59:11.929 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.928+0000 7f27f37d6640 1 -- start start 2024-04-25T10:59:11.931 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.929+0000 7f27f37d6640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f27ec0ce590 0x7f27ec15e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:11.931 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.929+0000 7f27f37d6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27ec0d2560 0x7f27ec0b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:11.931 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.929+0000 7f27f37d6640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27ec0b5f70 con 0x7f27ec0b5b60 2024-04-25T10:59:11.931 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.929+0000 7f27f37d6640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f27ec0d1e40 con 0x7f27ec0ce590 2024-04-25T10:59:11.931 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.929+0000 7f27f37d6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f27ec0d2970 con 0x7f27ec0d2560 2024-04-25T10:59:11.932 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.929+0000 7f27f1572640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f27ec0ce590 0x7f27ec15e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:11.932 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.930+0000 7f27f1d73640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f27ec0b5b60 0x7f27ec070170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:33664/0 (socket says 172.21.15.69:33664) 2024-04-25T10:59:11.932 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.930+0000 7f27f1572640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f27ec0ce590 0x7f27ec15e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55178/0 (socket says 172.21.15.69:55178) 2024-04-25T10:59:11.932 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.930+0000 7f27f1572640 1 -- 172.21.15.69:0/4031239432 learned_addr learned my addr 172.21.15.69:0/4031239432 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:11.932 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.930+0000 7f27f0d71640 1 --2- 172.21.15.69:0/4031239432 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27ec0d2560 0x7f27ec0b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:11.932 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.930+0000 7f27f1572640 1 -- 172.21.15.69:0/4031239432 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27ec0d2560 msgr2=0x7f27ec0b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:11.932 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.930+0000 7f27f1572640 1 --2- 172.21.15.69:0/4031239432 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27ec0d2560 0x7f27ec0b5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:11.932 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.930+0000 7f27f1572640 1 -- 172.21.15.69:0/4031239432 >> v1:172.21.15.69:6789/0 conn(0x7f27ec0b5b60 legacy=0x7f27ec070170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:11.932 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.930+0000 7f27f1572640 1 -- 172.21.15.69:0/4031239432 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27ec0d2c40 con 0x7f27ec0ce590 2024-04-25T10:59:11.932 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.931+0000 7f27f1572640 1 --2- 172.21.15.69:0/4031239432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27ec0ce590 0x7f27ec15e300 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f27e8009870 tx=0x7f27e802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4937dd98c2dc05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:11.932 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.931+0000 7f27e3fff640 1 -- 172.21.15.69:0/4031239432 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e8037070 con 0x7f27ec0ce590 2024-04-25T10:59:11.932 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.931+0000 7f27e3fff640 1 -- 172.21.15.69:0/4031239432 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27e8032440 con 0x7f27ec0ce590 2024-04-25T10:59:11.932 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.931+0000 7f27e3fff640 1 -- 172.21.15.69:0/4031239432 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e8032810 con 0x7f27ec0ce590 2024-04-25T10:59:11.933 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.931+0000 7f27f37d6640 1 -- 172.21.15.69:0/4031239432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27ec0ce590 msgr2=0x7f27ec15e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:11.934 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.931+0000 7f27f37d6640 1 --2- 172.21.15.69:0/4031239432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27ec0ce590 0x7f27ec15e300 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f27e8009870 tx=0x7f27e802a270 comp rx=0 tx=0).stop 2024-04-25T10:59:11.934 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.931+0000 7f27f37d6640 1 -- 172.21.15.69:0/4031239432 shutdown_connections 2024-04-25T10:59:11.934 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.931+0000 7f27f37d6640 1 --2- 172.21.15.69:0/4031239432 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27ec0d2560 0x7f27ec0b5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:11.934 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.931+0000 7f27f37d6640 1 --2- 172.21.15.69:0/4031239432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27ec0ce590 0x7f27ec15e300 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:11.934 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.931+0000 7f27f37d6640 1 -- 172.21.15.69:0/4031239432 >> 172.21.15.69:0/4031239432 conn(0x7f27ec072160 msgr2=0x7f27ec0d0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:11.934 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.932+0000 7f27f37d6640 1 -- 172.21.15.69:0/4031239432 shutdown_connections 2024-04-25T10:59:11.934 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.932+0000 7f27f37d6640 1 -- 172.21.15.69:0/4031239432 wait complete. 2024-04-25T10:59:11.934 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.933+0000 7f27f37d6640 1 Processor -- start 2024-04-25T10:59:11.934 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.933+0000 7f27f37d6640 1 -- start start 2024-04-25T10:59:11.937 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.935+0000 7f27f37d6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27ec0b5b60 0x7f27ec151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:11.937 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.935+0000 7f27f37d6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27ec0ce590 0x7f27ec0cac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:11.937 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.935+0000 7f27f37d6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27ec0d2560 0x7f27ec0cb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:11.937 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.935+0000 7f27f37d6640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec0d34e0 con 0x7f27ec0b5b60 2024-04-25T10:59:11.937 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.935+0000 7f27f37d6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec0d3200 con 0x7f27ec0d2560 2024-04-25T10:59:11.937 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.935+0000 7f27f37d6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f27ec0d37c0 con 0x7f27ec0ce590 2024-04-25T10:59:11.937 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.935+0000 7f27f1572640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27ec0b5b60 0x7f27ec151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:11.937 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.935+0000 7f27f1572640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27ec0b5b60 0x7f27ec151080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55182/0 (socket says 172.21.15.69:55182) 2024-04-25T10:59:11.937 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.935+0000 7f27f1572640 1 -- 172.21.15.69:0/3311706779 learned_addr learned my addr 172.21.15.69:0/3311706779 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:11.937 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.935+0000 7f27f1d73640 1 --2- 172.21.15.69:0/3311706779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27ec0d2560 0x7f27ec0cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:11.937 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.935+0000 7f27f0d71640 1 --2- 172.21.15.69:0/3311706779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27ec0ce590 0x7f27ec0cac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:11.937 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.935+0000 7f27f1572640 1 -- 172.21.15.69:0/3311706779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27ec0ce590 msgr2=0x7f27ec0cac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:11.937 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.935+0000 7f27f1572640 1 --2- 172.21.15.69:0/3311706779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27ec0ce590 0x7f27ec0cac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:11.938 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.935+0000 7f27f1572640 1 -- 172.21.15.69:0/3311706779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27ec0d2560 msgr2=0x7f27ec0cb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:11.938 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.935+0000 7f27f1572640 1 --2- 172.21.15.69:0/3311706779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27ec0d2560 0x7f27ec0cb160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:11.938 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.935+0000 7f27f1572640 1 -- 172.21.15.69:0/3311706779 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27ec0ccb30 con 0x7f27ec0b5b60 2024-04-25T10:59:11.938 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.935+0000 7f27f0d71640 1 --2- 172.21.15.69:0/3311706779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27ec0ce590 0x7f27ec0cac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:11.938 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.935+0000 7f27f1572640 1 --2- 172.21.15.69:0/3311706779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27ec0b5b60 0x7f27ec151080 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f27e8000c00 tx=0x7f27e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:11.939 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.935+0000 7f27e1ffb640 1 -- 172.21.15.69:0/3311706779 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e8041020 con 0x7f27ec0b5b60 2024-04-25T10:59:11.939 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.935+0000 7f27e1ffb640 1 -- 172.21.15.69:0/3311706779 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27e80306c0 con 0x7f27ec0b5b60 2024-04-25T10:59:11.939 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.935+0000 7f27e1ffb640 1 -- 172.21.15.69:0/3311706779 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e8037040 con 0x7f27ec0b5b60 2024-04-25T10:59:11.940 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.935+0000 7f27f37d6640 1 -- 172.21.15.69:0/3311706779 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27ec0cce10 con 0x7f27ec0b5b60 2024-04-25T10:59:11.940 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.936+0000 7f27f37d6640 1 -- 172.21.15.69:0/3311706779 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27ec0c9930 con 0x7f27ec0b5b60 2024-04-25T10:59:11.941 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.937+0000 7f27f37d6640 1 -- 172.21.15.69:0/3311706779 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27ec06a160 con 0x7f27ec0b5b60 2024-04-25T10:59:11.941 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.937+0000 7f27e1ffb640 1 -- 172.21.15.69:0/3311706779 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f27e8044020 con 0x7f27ec0b5b60 2024-04-25T10:59:11.941 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.938+0000 7f27e1ffb640 1 --2- 172.21.15.69:0/3311706779 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f27d00744b0 0x7f27d00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:11.941 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.938+0000 7f27e1ffb640 1 -- 172.21.15.69:0/3311706779 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f27e80b5770 con 0x7f27ec0b5b60 2024-04-25T10:59:11.941 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.938+0000 7f27f0d71640 1 --2- 172.21.15.69:0/3311706779 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f27d00744b0 0x7f27d00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:11.941 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.938+0000 7f27f0d71640 1 --2- 172.21.15.69:0/3311706779 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f27d00744b0 0x7f27d00769c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f27dc006fd0 tx=0x7f27dc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:11.943 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:11.943+0000 7f27e1ffb640 1 -- 172.21.15.69:0/3311706779 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f27e804b030 con 0x7f27ec0b5b60 2024-04-25T10:59:12.119 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:12.119+0000 7f27f37d6640 1 -- 172.21.15.69:0/3311706779 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f27ec0b3fd0 con 0x7f27ec0b5b60 2024-04-25T10:59:12.120 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:12.120+0000 7f27e1ffb640 1 -- 172.21.15.69:0/3311706779 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12884 (secure 0 0 0) 0x7f27e8080250 con 0x7f27ec0b5b60 2024-04-25T10:59:12.120 INFO:teuthology.misc.health.smithi069.stdout: 2024-04-25T10:59:12.121 INFO:teuthology.misc.health.smithi069.stdout:{"epoch":16,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T10:59:08.565477+0000","last_up_change":"2024-04-25T10:59:05.546839+0000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:59:06.422674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6825","nonce":1876101065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6827","nonce":1876101065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6831","nonce":1876101065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6829","nonce":1876101065}]},"public_addr":"172.21.15.69:6825/1876101065","cluster_addr":"172.21.15.69:6827/1876101065","heartbeat_back_addr":"172.21.15.69:6831/1876101065","heartbeat_front_addr":"172.21.15.69:6829/1876101065","state":["exists","up"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6801","nonce":1806438563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6803","nonce":1806438563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6807","nonce":1806438563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6805","nonce":1806438563}]},"public_addr":"172.21.15.69:6801/1806438563","cluster_addr":"172.21.15.69:6803/1806438563","heartbeat_back_addr":"172.21.15.69:6807/1806438563","heartbeat_front_addr":"172.21.15.69:6805/1806438563","state":["exists","up"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6817","nonce":1593617983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6819","nonce":1593617983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6823","nonce":1593617983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6821","nonce":1593617983}]},"public_addr":"172.21.15.69:6817/1593617983","cluster_addr":"172.21.15.69:6819/1593617983","heartbeat_back_addr":"172.21.15.69:6823/1593617983","heartbeat_front_addr":"172.21.15.69:6821/1593617983","state":["exists","up"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6809","nonce":3950661294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6811","nonce":3950661294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6815","nonce":3950661294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6813","nonce":3950661294}]},"public_addr":"172.21.15.69:6809/3950661294","cluster_addr":"172.21.15.69:6811/3950661294","heartbeat_back_addr":"172.21.15.69:6815/3950661294","heartbeat_front_addr":"172.21.15.69:6813/3950661294","state":["exists","up"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6801","nonce":2508929872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6803","nonce":2508929872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6807","nonce":2508929872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6805","nonce":2508929872}]},"public_addr":"172.21.15.146:6801/2508929872","cluster_addr":"172.21.15.146:6803/2508929872","heartbeat_back_addr":"172.21.15.146:6807/2508929872","heartbeat_front_addr":"172.21.15.146:6805/2508929872","state":["exists","up"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6811","nonce":1310911138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6816","nonce":1310911138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6823","nonce":1310911138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6821","nonce":1310911138}]},"public_addr":"172.21.15.146:6811/1310911138","cluster_addr":"172.21.15.146:6816/1310911138","heartbeat_back_addr":"172.21.15.146:6823/1310911138","heartbeat_front_addr":"172.21.15.146:6821/1310911138","state":["exists","up"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6810","nonce":634688021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6813","nonce":634688021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6820","nonce":634688021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6817","nonce":634688021}]},"public_addr":"172.21.15.146:6810/634688021","cluster_addr":"172.21.15.146:6813/634688021","heartbeat_back_addr":"172.21.15.146:6820/634688021","heartbeat_front_addr":"172.21.15.146:6817/634688021","state":["exists","up"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6825","nonce":3814215935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6827","nonce":3814215935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6831","nonce":3814215935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6829","nonce":3814215935}]},"public_addr":"172.21.15.146:6825/3814215935","cluster_addr":"172.21.15.146:6827/3814215935","heartbeat_back_addr":"172.21.15.146:6831/3814215935","heartbeat_front_addr":"172.21.15.146:6829/3814215935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.820200+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.850189+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.764560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.866248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.807897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.869881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.782310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.827131+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:59:12.123 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:12.122+0000 7f27f37d6640 1 -- 172.21.15.69:0/3311706779 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f27d00744b0 msgr2=0x7f27d00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:12.123 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:12.122+0000 7f27f37d6640 1 --2- 172.21.15.69:0/3311706779 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f27d00744b0 0x7f27d00769c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f27dc006fd0 tx=0x7f27dc008040 comp rx=0 tx=0).stop 2024-04-25T10:59:12.123 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:12.122+0000 7f27f37d6640 1 -- 172.21.15.69:0/3311706779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27ec0b5b60 msgr2=0x7f27ec151080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:12.123 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:12.122+0000 7f27f37d6640 1 --2- 172.21.15.69:0/3311706779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27ec0b5b60 0x7f27ec151080 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f27e8000c00 tx=0x7f27e80046c0 comp rx=0 tx=0).stop 2024-04-25T10:59:12.123 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:12.122+0000 7f27f37d6640 1 -- 172.21.15.69:0/3311706779 shutdown_connections 2024-04-25T10:59:12.123 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:12.122+0000 7f27f37d6640 1 --2- 172.21.15.69:0/3311706779 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f27d00744b0 0x7f27d00769c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:12.123 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:12.122+0000 7f27f37d6640 1 --2- 172.21.15.69:0/3311706779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27ec0d2560 0x7f27ec0cb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:12.123 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:12.122+0000 7f27f37d6640 1 --2- 172.21.15.69:0/3311706779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27ec0ce590 0x7f27ec0cac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:12.123 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:12.122+0000 7f27f37d6640 1 --2- 172.21.15.69:0/3311706779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27ec0b5b60 0x7f27ec151080 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:12.123 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:12.122+0000 7f27f37d6640 1 -- 172.21.15.69:0/3311706779 >> 172.21.15.69:0/3311706779 conn(0x7f27ec072160 msgr2=0x7f27ec15c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:12.123 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:12.122+0000 7f27f37d6640 1 -- 172.21.15.69:0/3311706779 shutdown_connections 2024-04-25T10:59:12.123 INFO:teuthology.misc.health.smithi069.stderr:2024-04-25T10:59:12.122+0000 7f27f37d6640 1 -- 172.21.15.69:0/3311706779 wait complete. 2024-04-25T10:59:12.132 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T10:59:12.133 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T10:59:12.133 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:59:12.133 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:59:12.136 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:59:12.136 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:59:12.136 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:59:12.138 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:59:12.138 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:59:12.138 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:59:12.141 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:59:12.141 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:59:12.141 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:59:12.143 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:59:12.143 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T10:59:12.144 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:59:12.220 INFO:tasks.ceph.mds.a.smithi069.stdout:starting mds.a at 2024-04-25T10:59:12.221 INFO:tasks.ceph.mds.c.smithi069.stdout:starting mds.c at 2024-04-25T10:59:12.221 INFO:tasks.ceph.mds.a.smithi069.stderr:2024-04-25T10:59:12.220+0000 7f28fc09e640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-04-25T10:59:12.237 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2024-04-25T10:59:12.240 INFO:tasks.ceph.mds.b.smithi146.stdout:starting mds.b at 2024-04-25T10:59:12.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.258+0000 7fcf22861640 1 Processor -- start 2024-04-25T10:59:12.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.258+0000 7fcf22861640 1 -- start start 2024-04-25T10:59:12.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.259+0000 7fcf22861640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcf1c0b7f50 0x7fcf1c0b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:12.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.259+0000 7fcf22861640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf1c0b28b0 con 0x7fcf1c0cf110 2024-04-25T10:59:12.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.259+0000 7fcf22861640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf1c0b2b90 con 0x7fcf1c15e480 2024-04-25T10:59:12.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.259+0000 7fcf22861640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fcf1c0b34c0 con 0x7fcf1c0b7f50 2024-04-25T10:59:12.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.259+0000 7fcf1b7fe640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fcf1c0cf110 0x7fcf1c0b7840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:33670/0 (socket says 172.21.15.69:33670) 2024-04-25T10:59:12.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.259+0000 7fcf1b7fe640 1 -- 172.21.15.69:0/2971735817 learned_addr learned my addr 172.21.15.69:0/2971735817 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:12.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.259+0000 7fcf20dfe640 1 --2- 172.21.15.69:0/2971735817 >> v2:172.21.15.146:3300/0 conn(0x7fcf1c0b7f50 0x7fcf1c0b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:12.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.259+0000 7fcf1affd640 1 -- 172.21.15.69:0/2971735817 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1447777514 0 0) 0x7fcf1c0b28b0 con 0x7fcf1c0cf110 2024-04-25T10:59:12.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.259+0000 7fcf1affd640 1 -- 172.21.15.69:0/2971735817 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf04003520 con 0x7fcf1c0cf110 2024-04-25T10:59:12.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.260+0000 7fcf1affd640 1 -- 172.21.15.69:0/2971735817 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 605687951 0 0) 0x7fcf04003520 con 0x7fcf1c0cf110 2024-04-25T10:59:12.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.260+0000 7fcf1affd640 1 -- 172.21.15.69:0/2971735817 >> v2:172.21.15.146:3300/0 conn(0x7fcf1c0b7f50 msgr2=0x7fcf1c0b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:12.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.260+0000 7fcf1affd640 1 --2- 172.21.15.69:0/2971735817 >> v2:172.21.15.146:3300/0 conn(0x7fcf1c0b7f50 0x7fcf1c0b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:12.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.260+0000 7fcf1affd640 1 -- 172.21.15.69:0/2971735817 >> v1:172.21.15.146:6789/0 conn(0x7fcf1c15e480 legacy=0x7fcf1c0af7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:12.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.260+0000 7fcf1affd640 1 -- 172.21.15.69:0/2971735817 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1c0b2e70 con 0x7fcf1c0cf110 2024-04-25T10:59:12.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.260+0000 7fcf1affd640 1 -- 172.21.15.69:0/2971735817 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7fcf0c001670 con 0x7fcf1c0cf110 2024-04-25T10:59:12.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.260+0000 7fcf1affd640 1 -- 172.21.15.69:0/2971735817 >> v1:172.21.15.69:6789/0 conn(0x7fcf1c0cf110 legacy=0x7fcf1c0b7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:12.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.260+0000 7fcf1affd640 1 --2- 172.21.15.69:0/2971735817 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf040039b0 0x7fcf04003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:12.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.260+0000 7fcf1affd640 1 --2- 172.21.15.69:0/2971735817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf04004800 0x7fcf04004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:12.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.260+0000 7fcf1affd640 1 --2- 172.21.15.69:0/2971735817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf04005270 0x7fcf0400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:12.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.260+0000 7fcf1affd640 1 -- 172.21.15.69:0/2971735817 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c0b34c0 con 0x7fcf04004800 2024-04-25T10:59:12.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.260+0000 7fcf1affd640 1 -- 172.21.15.69:0/2971735817 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0400c190 con 0x7fcf04005270 2024-04-25T10:59:12.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.260+0000 7fcf1affd640 1 -- 172.21.15.69:0/2971735817 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fcf0400c390 con 0x7fcf040039b0 2024-04-25T10:59:12.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.260+0000 7fcf1b7fe640 1 --2- 172.21.15.69:0/2971735817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf04004800 0x7fcf04004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:12.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.260+0000 7fcf1bfff640 1 --2- 172.21.15.69:0/2971735817 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf040039b0 0x7fcf04003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:12.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.260+0000 7fcf20dfe640 1 --2- 172.21.15.69:0/2971735817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf04005270 0x7fcf0400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:12.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.260+0000 7fcf1b7fe640 1 -- 172.21.15.69:0/2971735817 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf040039b0 msgr2=0x7fcf04003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:12.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.260+0000 7fcf1b7fe640 1 --2- 172.21.15.69:0/2971735817 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf040039b0 0x7fcf04003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:12.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.260+0000 7fcf1b7fe640 1 -- 172.21.15.69:0/2971735817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf04005270 msgr2=0x7fcf0400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:12.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.260+0000 7fcf1b7fe640 1 --2- 172.21.15.69:0/2971735817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf04005270 0x7fcf0400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:12.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.260+0000 7fcf1b7fe640 1 -- 172.21.15.69:0/2971735817 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf04003520 con 0x7fcf04004800 2024-04-25T10:59:12.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.260+0000 7fcf20dfe640 1 --2- 172.21.15.69:0/2971735817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf04005270 0x7fcf0400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:12.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.260+0000 7fcf20dfe640 1 -- 172.21.15.69:0/2971735817 reap_dead start 2024-04-25T10:59:12.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.261+0000 7fcf1b7fe640 1 --2- 172.21.15.69:0/2971735817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf04004800 0x7fcf04004c10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00c240 tx=0x7fcf04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e414fc40ab29a83c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:12.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.261+0000 7fcf1affd640 1 -- 172.21.15.69:0/2971735817 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c007cf0 con 0x7fcf04004800 2024-04-25T10:59:12.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.261+0000 7fcf1affd640 1 -- 172.21.15.69:0/2971735817 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf0c005af0 con 0x7fcf04004800 2024-04-25T10:59:12.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.261+0000 7fcf1affd640 1 -- 172.21.15.69:0/2971735817 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c007070 con 0x7fcf04004800 2024-04-25T10:59:12.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.262+0000 7fcf22861640 1 -- 172.21.15.69:0/2971735817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf04004800 msgr2=0x7fcf04004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:12.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.262+0000 7fcf22861640 1 --2- 172.21.15.69:0/2971735817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf04004800 0x7fcf04004c10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00c240 tx=0x7fcf04005830 comp rx=0 tx=0).stop 2024-04-25T10:59:12.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.262+0000 7fcf22861640 1 -- 172.21.15.69:0/2971735817 shutdown_connections 2024-04-25T10:59:12.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.262+0000 7fcf22861640 1 --2- 172.21.15.69:0/2971735817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf04004800 0x7fcf04004c10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:12.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.262+0000 7fcf22861640 1 -- 172.21.15.69:0/2971735817 >> 172.21.15.69:0/2971735817 conn(0x7fcf1c09b7d0 msgr2=0x7fcf1c09bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:12.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.262+0000 7fcf22861640 1 -- 172.21.15.69:0/2971735817 shutdown_connections 2024-04-25T10:59:12.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.262+0000 7fcf22861640 1 -- 172.21.15.69:0/2971735817 wait complete. 2024-04-25T10:59:12.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.264+0000 7fcf22861640 1 Processor -- start 2024-04-25T10:59:12.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.264+0000 7fcf22861640 1 -- start start 2024-04-25T10:59:12.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.265+0000 7fcf22861640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf04004800 0x7fcf1c1f3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:12.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.265+0000 7fcf22861640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf1c0cc360 0x7fcf1c159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:12.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.265+0000 7fcf22861640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf1c0cc770 0x7fcf1c06d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:12.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.265+0000 7fcf22861640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c0b25c0 con 0x7fcf04004800 2024-04-25T10:59:12.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.265+0000 7fcf22861640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c0b2440 con 0x7fcf1c0cc770 2024-04-25T10:59:12.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.265+0000 7fcf22861640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fcf1c15e480 con 0x7fcf1c0cc360 2024-04-25T10:59:12.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.265+0000 7fcf1bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf04004800 0x7fcf1c1f3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:12.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.265+0000 7fcf1bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf04004800 0x7fcf1c1f3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55220/0 (socket says 172.21.15.69:55220) 2024-04-25T10:59:12.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.265+0000 7fcf1bfff640 1 -- 172.21.15.69:0/2086483647 learned_addr learned my addr 172.21.15.69:0/2086483647 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:12.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.265+0000 7fcf20dfe640 1 --2- 172.21.15.69:0/2086483647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf1c0cc770 0x7fcf1c06d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:12.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.265+0000 7fcf1b7fe640 1 --2- 172.21.15.69:0/2086483647 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf1c0cc360 0x7fcf1c159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:12.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.265+0000 7fcf1bfff640 1 -- 172.21.15.69:0/2086483647 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf1c0cc360 msgr2=0x7fcf1c159fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:12.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.265+0000 7fcf1bfff640 1 --2- 172.21.15.69:0/2086483647 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf1c0cc360 0x7fcf1c159fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:12.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.265+0000 7fcf1bfff640 1 -- 172.21.15.69:0/2086483647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf1c0cc770 msgr2=0x7fcf1c06d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:12.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.265+0000 7fcf1bfff640 1 --2- 172.21.15.69:0/2086483647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf1c0cc770 0x7fcf1c06d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:12.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.265+0000 7fcf1bfff640 1 -- 172.21.15.69:0/2086483647 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1c15fbc0 con 0x7fcf04004800 2024-04-25T10:59:12.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.265+0000 7fcf1b7fe640 1 --2- 172.21.15.69:0/2086483647 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf1c0cc360 0x7fcf1c159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:12.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.266+0000 7fcf1bfff640 1 --2- 172.21.15.69:0/2086483647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf04004800 0x7fcf1c1f3a80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000f820 tx=0x7fcf1000fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:12.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.266+0000 7fcf18ff9640 1 -- 172.21.15.69:0/2086483647 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf100050d0 con 0x7fcf04004800 2024-04-25T10:59:12.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.266+0000 7fcf18ff9640 1 -- 172.21.15.69:0/2086483647 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf10018070 con 0x7fcf04004800 2024-04-25T10:59:12.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.266+0000 7fcf22861640 1 -- 172.21.15.69:0/2086483647 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf1c15fea0 con 0x7fcf04004800 2024-04-25T10:59:12.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.266+0000 7fcf18ff9640 1 -- 172.21.15.69:0/2086483647 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf10005490 con 0x7fcf04004800 2024-04-25T10:59:12.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.266+0000 7fcf22861640 1 -- 172.21.15.69:0/2086483647 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf1c160460 con 0x7fcf04004800 2024-04-25T10:59:12.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.267+0000 7fcf22861640 1 -- 172.21.15.69:0/2086483647 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf1c050740 con 0x7fcf04004800 2024-04-25T10:59:12.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.267+0000 7fcf18ff9640 1 -- 172.21.15.69:0/2086483647 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcf1000d730 con 0x7fcf04004800 2024-04-25T10:59:12.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.268+0000 7fcf18ff9640 1 --2- 172.21.15.69:0/2086483647 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fcf080744e0 0x7fcf080769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:12.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.268+0000 7fcf18ff9640 1 -- 172.21.15.69:0/2086483647 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fcf10094c60 con 0x7fcf04004800 2024-04-25T10:59:12.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.268+0000 7fcf1b7fe640 1 --2- 172.21.15.69:0/2086483647 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fcf080744e0 0x7fcf080769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:12.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.268+0000 7fcf1b7fe640 1 --2- 172.21.15.69:0/2086483647 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fcf080744e0 0x7fcf080769f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c032000 tx=0x7fcf0c009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:12.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.272+0000 7fcf18ff9640 1 -- 172.21.15.69:0/2086483647 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcf1005f7d0 con 0x7fcf04004800 2024-04-25T10:59:12.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.448+0000 7fcf22861640 1 -- 172.21.15.69:0/2086483647 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf1c0b6f30 con 0x7fcf04004800 2024-04-25T10:59:12.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.449+0000 7fcf18ff9640 1 -- 172.21.15.69:0/2086483647 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12884 (secure 0 0 0) 0x7fcf1005f0d0 con 0x7fcf04004800 2024-04-25T10:59:12.449 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:12.450 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":16,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T10:59:08.565477+0000","last_up_change":"2024-04-25T10:59:05.546839+0000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:59:06.422674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6825","nonce":1876101065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6827","nonce":1876101065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6831","nonce":1876101065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6829","nonce":1876101065}]},"public_addr":"172.21.15.69:6825/1876101065","cluster_addr":"172.21.15.69:6827/1876101065","heartbeat_back_addr":"172.21.15.69:6831/1876101065","heartbeat_front_addr":"172.21.15.69:6829/1876101065","state":["exists","up"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6801","nonce":1806438563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6803","nonce":1806438563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6807","nonce":1806438563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6805","nonce":1806438563}]},"public_addr":"172.21.15.69:6801/1806438563","cluster_addr":"172.21.15.69:6803/1806438563","heartbeat_back_addr":"172.21.15.69:6807/1806438563","heartbeat_front_addr":"172.21.15.69:6805/1806438563","state":["exists","up"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6817","nonce":1593617983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6819","nonce":1593617983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6823","nonce":1593617983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6821","nonce":1593617983}]},"public_addr":"172.21.15.69:6817/1593617983","cluster_addr":"172.21.15.69:6819/1593617983","heartbeat_back_addr":"172.21.15.69:6823/1593617983","heartbeat_front_addr":"172.21.15.69:6821/1593617983","state":["exists","up"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6809","nonce":3950661294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6811","nonce":3950661294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6815","nonce":3950661294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6813","nonce":3950661294}]},"public_addr":"172.21.15.69:6809/3950661294","cluster_addr":"172.21.15.69:6811/3950661294","heartbeat_back_addr":"172.21.15.69:6815/3950661294","heartbeat_front_addr":"172.21.15.69:6813/3950661294","state":["exists","up"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6801","nonce":2508929872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6803","nonce":2508929872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6807","nonce":2508929872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6805","nonce":2508929872}]},"public_addr":"172.21.15.146:6801/2508929872","cluster_addr":"172.21.15.146:6803/2508929872","heartbeat_back_addr":"172.21.15.146:6807/2508929872","heartbeat_front_addr":"172.21.15.146:6805/2508929872","state":["exists","up"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6811","nonce":1310911138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6816","nonce":1310911138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6823","nonce":1310911138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6821","nonce":1310911138}]},"public_addr":"172.21.15.146:6811/1310911138","cluster_addr":"172.21.15.146:6816/1310911138","heartbeat_back_addr":"172.21.15.146:6823/1310911138","heartbeat_front_addr":"172.21.15.146:6821/1310911138","state":["exists","up"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6810","nonce":634688021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6813","nonce":634688021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6820","nonce":634688021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6817","nonce":634688021}]},"public_addr":"172.21.15.146:6810/634688021","cluster_addr":"172.21.15.146:6813/634688021","heartbeat_back_addr":"172.21.15.146:6820/634688021","heartbeat_front_addr":"172.21.15.146:6817/634688021","state":["exists","up"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6825","nonce":3814215935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6827","nonce":3814215935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6831","nonce":3814215935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6829","nonce":3814215935}]},"public_addr":"172.21.15.146:6825/3814215935","cluster_addr":"172.21.15.146:6827/3814215935","heartbeat_back_addr":"172.21.15.146:6831/3814215935","heartbeat_front_addr":"172.21.15.146:6829/3814215935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.820200+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.850189+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.764560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.866248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.807897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.869881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.782310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.827131+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:59:12.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.451+0000 7fcf22861640 1 -- 172.21.15.69:0/2086483647 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fcf080744e0 msgr2=0x7fcf080769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:12.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.451+0000 7fcf22861640 1 --2- 172.21.15.69:0/2086483647 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fcf080744e0 0x7fcf080769f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c032000 tx=0x7fcf0c009030 comp rx=0 tx=0).stop 2024-04-25T10:59:12.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.451+0000 7fcf22861640 1 -- 172.21.15.69:0/2086483647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf04004800 msgr2=0x7fcf1c1f3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:12.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.451+0000 7fcf22861640 1 --2- 172.21.15.69:0/2086483647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf04004800 0x7fcf1c1f3a80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000f820 tx=0x7fcf1000fce0 comp rx=0 tx=0).stop 2024-04-25T10:59:12.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.451+0000 7fcf22861640 1 -- 172.21.15.69:0/2086483647 shutdown_connections 2024-04-25T10:59:12.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.451+0000 7fcf22861640 1 --2- 172.21.15.69:0/2086483647 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fcf080744e0 0x7fcf080769f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:12.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.451+0000 7fcf22861640 1 --2- 172.21.15.69:0/2086483647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf1c0cc770 0x7fcf1c06d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:12.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.451+0000 7fcf22861640 1 --2- 172.21.15.69:0/2086483647 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf1c0cc360 0x7fcf1c159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:12.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.451+0000 7fcf22861640 1 --2- 172.21.15.69:0/2086483647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf04004800 0x7fcf1c1f3a80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:12.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.451+0000 7fcf22861640 1 -- 172.21.15.69:0/2086483647 >> 172.21.15.69:0/2086483647 conn(0x7fcf1c09b7d0 msgr2=0x7fcf1c15ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:12.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.451+0000 7fcf22861640 1 -- 172.21.15.69:0/2086483647 shutdown_connections 2024-04-25T10:59:12.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.451+0000 7fcf22861640 1 -- 172.21.15.69:0/2086483647 wait complete. 2024-04-25T10:59:12.463 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:59:06.422674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:59:12.463 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:59:12.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.573+0000 7f1377127640 1 Processor -- start 2024-04-25T10:59:12.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.573+0000 7f1377127640 1 -- start start 2024-04-25T10:59:12.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.575+0000 7f1377127640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f1370164b00 0x7f13700b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:12.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.575+0000 7f1377127640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1370160fa0 0x7f13700b80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:12.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.575+0000 7f1377127640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13701615f0 con 0x7f137015df80 2024-04-25T10:59:12.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.575+0000 7f1377127640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f13701613b0 con 0x7f1370164b00 2024-04-25T10:59:12.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.575+0000 7f1377127640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1370164670 con 0x7f1370160fa0 2024-04-25T10:59:12.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.575+0000 7f1374ec3640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f1370164b00 0x7f13700b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:12.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.575+0000 7f1374ec3640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f1370164b00 0x7f13700b7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55230/0 (socket says 172.21.15.69:55230) 2024-04-25T10:59:12.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.575+0000 7f136ffff640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f137015df80 0x7f1370160890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:33674/0 (socket says 172.21.15.69:33674) 2024-04-25T10:59:12.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.575+0000 7f1374ec3640 1 -- 172.21.15.69:0/1267045247 learned_addr learned my addr 172.21.15.69:0/1267045247 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:12.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.575+0000 7f13756c4640 1 --2- 172.21.15.69:0/1267045247 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1370160fa0 0x7f13700b80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:12.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.575+0000 7f1374ec3640 1 -- 172.21.15.69:0/1267045247 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1370160fa0 msgr2=0x7f13700b80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:12.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.575+0000 7f1374ec3640 1 --2- 172.21.15.69:0/1267045247 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1370160fa0 0x7f13700b80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:12.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.575+0000 7f1374ec3640 1 -- 172.21.15.69:0/1267045247 >> v1:172.21.15.69:6789/0 conn(0x7f137015df80 legacy=0x7f1370160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:12.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.575+0000 7f1374ec3640 1 -- 172.21.15.69:0/1267045247 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1370164020 con 0x7f1370164b00 2024-04-25T10:59:12.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.575+0000 7f136f7fe640 1 -- 172.21.15.69:0/1267045247 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2273792118 0 0) 0x7f13701615f0 con 0x7f137015df80 2024-04-25T10:59:12.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.576+0000 7f1374ec3640 1 --2- 172.21.15.69:0/1267045247 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1370164b00 0x7f13700b7b80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1360009870 tx=0x7f136002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b1133790059960 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:12.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.576+0000 7f136f7fe640 1 -- 172.21.15.69:0/1267045247 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1360037070 con 0x7f1370164b00 2024-04-25T10:59:12.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.576+0000 7f136f7fe640 1 -- 172.21.15.69:0/1267045247 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1360032440 con 0x7f1370164b00 2024-04-25T10:59:12.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.576+0000 7f136f7fe640 1 -- 172.21.15.69:0/1267045247 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1360032810 con 0x7f1370164b00 2024-04-25T10:59:12.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.576+0000 7f1377127640 1 -- 172.21.15.69:0/1267045247 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1370164b00 msgr2=0x7f13700b7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:12.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.576+0000 7f1377127640 1 --2- 172.21.15.69:0/1267045247 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1370164b00 0x7f13700b7b80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1360009870 tx=0x7f136002a270 comp rx=0 tx=0).stop 2024-04-25T10:59:12.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.577+0000 7f1377127640 1 -- 172.21.15.69:0/1267045247 shutdown_connections 2024-04-25T10:59:12.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.577+0000 7f1377127640 1 --2- 172.21.15.69:0/1267045247 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1370160fa0 0x7f13700b80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:12.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.577+0000 7f1377127640 1 --2- 172.21.15.69:0/1267045247 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1370164b00 0x7f13700b7b80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:12.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.577+0000 7f1377127640 1 -- 172.21.15.69:0/1267045247 >> 172.21.15.69:0/1267045247 conn(0x7f13700d2bc0 msgr2=0x7f13700d2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:12.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.577+0000 7f1377127640 1 -- 172.21.15.69:0/1267045247 shutdown_connections 2024-04-25T10:59:12.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.577+0000 7f1377127640 1 -- 172.21.15.69:0/1267045247 wait complete. 2024-04-25T10:59:12.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.578+0000 7f1377127640 1 Processor -- start 2024-04-25T10:59:12.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.578+0000 7f1377127640 1 -- start start 2024-04-25T10:59:12.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.580+0000 7f1377127640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f137015df80 0x7f1370156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:12.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.580+0000 7f1377127640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1370160fa0 0x7f137015ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:12.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.580+0000 7f1377127640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1370164b00 0x7f1370155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:12.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.580+0000 7f1377127640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f13700b4a00 con 0x7f1370164b00 2024-04-25T10:59:12.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.580+0000 7f1377127640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f13700b4880 con 0x7f1370160fa0 2024-04-25T10:59:12.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.580+0000 7f1377127640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f13700b4b80 con 0x7f137015df80 2024-04-25T10:59:12.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.580+0000 7f13756c4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1370164b00 0x7f1370155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:12.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.580+0000 7f13756c4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1370164b00 0x7f1370155a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55236/0 (socket says 172.21.15.69:55236) 2024-04-25T10:59:12.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.580+0000 7f13756c4640 1 -- 172.21.15.69:0/2732138798 learned_addr learned my addr 172.21.15.69:0/2732138798 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:12.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.580+0000 7f1374ec3640 1 --2- 172.21.15.69:0/2732138798 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f137015df80 0x7f1370156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:12.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.580+0000 7f136ffff640 1 --2- 172.21.15.69:0/2732138798 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1370160fa0 0x7f137015ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:12.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.580+0000 7f13756c4640 1 -- 172.21.15.69:0/2732138798 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f137015df80 msgr2=0x7f1370156370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:12.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.580+0000 7f13756c4640 1 --2- 172.21.15.69:0/2732138798 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f137015df80 0x7f1370156370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:12.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.580+0000 7f13756c4640 1 -- 172.21.15.69:0/2732138798 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1370160fa0 msgr2=0x7f137015ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:12.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.580+0000 7f13756c4640 1 --2- 172.21.15.69:0/2732138798 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1370160fa0 0x7f137015ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:12.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.580+0000 7f13756c4640 1 -- 172.21.15.69:0/2732138798 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f137015b240 con 0x7f1370164b00 2024-04-25T10:59:12.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.580+0000 7f136ffff640 1 --2- 172.21.15.69:0/2732138798 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1370160fa0 0x7f137015ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:12.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.580+0000 7f13756c4640 1 --2- 172.21.15.69:0/2732138798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1370164b00 0x7f1370155a10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f136800efc0 tx=0x7f136800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:12.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.580+0000 7f136d7fa640 1 -- 172.21.15.69:0/2732138798 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1368018020 con 0x7f1370164b00 2024-04-25T10:59:12.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.580+0000 7f136d7fa640 1 -- 172.21.15.69:0/2732138798 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1368014480 con 0x7f1370164b00 2024-04-25T10:59:12.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.580+0000 7f1377127640 1 -- 172.21.15.69:0/2732138798 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1370159c60 con 0x7f1370164b00 2024-04-25T10:59:12.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.580+0000 7f136d7fa640 1 -- 172.21.15.69:0/2732138798 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1368009e60 con 0x7f1370164b00 2024-04-25T10:59:12.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.581+0000 7f1377127640 1 -- 172.21.15.69:0/2732138798 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f137015a220 con 0x7f1370164b00 2024-04-25T10:59:12.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.582+0000 7f1377127640 1 -- 172.21.15.69:0/2732138798 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1370163700 con 0x7f1370164b00 2024-04-25T10:59:12.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.582+0000 7f136d7fa640 1 -- 172.21.15.69:0/2732138798 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1368005d10 con 0x7f1370164b00 2024-04-25T10:59:12.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.582+0000 7f136d7fa640 1 --2- 172.21.15.69:0/2732138798 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f135c0744b0 0x7f135c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:12.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.582+0000 7f136d7fa640 1 -- 172.21.15.69:0/2732138798 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f1368095c30 con 0x7f1370164b00 2024-04-25T10:59:12.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.583+0000 7f1374ec3640 1 --2- 172.21.15.69:0/2732138798 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f135c0744b0 0x7f135c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:12.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.583+0000 7f1374ec3640 1 --2- 172.21.15.69:0/2732138798 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f135c0744b0 0x7f135c0769c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1360034000 tx=0x7f136002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:12.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.587+0000 7f136d7fa640 1 -- 172.21.15.69:0/2732138798 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1370163700 con 0x7f1370164b00 2024-04-25T10:59:12.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.764+0000 7f1377127640 1 -- 172.21.15.69:0/2732138798 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1370160ca0 con 0x7f1370164b00 2024-04-25T10:59:12.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.765+0000 7f136d7fa640 1 -- 172.21.15.69:0/2732138798 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f1368022030 con 0x7f1370164b00 2024-04-25T10:59:12.765 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-25T10:59:12.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.767+0000 7f1377127640 1 -- 172.21.15.69:0/2732138798 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f135c0744b0 msgr2=0x7f135c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:12.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.767+0000 7f1377127640 1 --2- 172.21.15.69:0/2732138798 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f135c0744b0 0x7f135c0769c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1360034000 tx=0x7f136002e000 comp rx=0 tx=0).stop 2024-04-25T10:59:12.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.767+0000 7f1377127640 1 -- 172.21.15.69:0/2732138798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1370164b00 msgr2=0x7f1370155a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:12.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.767+0000 7f1377127640 1 --2- 172.21.15.69:0/2732138798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1370164b00 0x7f1370155a10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f136800efc0 tx=0x7f136800c490 comp rx=0 tx=0).stop 2024-04-25T10:59:12.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.767+0000 7f1377127640 1 -- 172.21.15.69:0/2732138798 shutdown_connections 2024-04-25T10:59:12.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.767+0000 7f1377127640 1 --2- 172.21.15.69:0/2732138798 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f135c0744b0 0x7f135c0769c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:12.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.767+0000 7f1377127640 1 --2- 172.21.15.69:0/2732138798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1370164b00 0x7f1370155a10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:12.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.767+0000 7f1377127640 1 --2- 172.21.15.69:0/2732138798 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1370160fa0 0x7f137015ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:12.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.767+0000 7f1377127640 1 --2- 172.21.15.69:0/2732138798 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f137015df80 0x7f1370156370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:12.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.767+0000 7f1377127640 1 -- 172.21.15.69:0/2732138798 >> 172.21.15.69:0/2732138798 conn(0x7f13700d2bc0 msgr2=0x7f137015e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:12.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.767+0000 7f1377127640 1 -- 172.21.15.69:0/2732138798 shutdown_connections 2024-04-25T10:59:12.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.767+0000 7f1377127640 1 -- 172.21.15.69:0/2732138798 wait complete. 2024-04-25T10:59:12.779 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:59:12.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.885+0000 7f0e487ba640 1 Processor -- start 2024-04-25T10:59:12.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.885+0000 7f0e487ba640 1 -- start start 2024-04-25T10:59:12.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.886+0000 7f0e487ba640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0e400b8ca0 0x7f0e40163fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:12.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.886+0000 7f0e487ba640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e401644e0 0x7f0e401648f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:12.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.886+0000 7f0e487ba640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e4006ed60 con 0x7f0e400d1f30 2024-04-25T10:59:12.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.886+0000 7f0e487ba640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f0e400d35a0 con 0x7f0e400b8ca0 2024-04-25T10:59:12.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.886+0000 7f0e487ba640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0e4006e8d0 con 0x7f0e401644e0 2024-04-25T10:59:12.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.886+0000 7f0e46556640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0e400b8ca0 0x7f0e40163fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:12.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.886+0000 7f0e46556640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0e400b8ca0 0x7f0e40163fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55242/0 (socket says 172.21.15.69:55242) 2024-04-25T10:59:12.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.886+0000 7f0e46556640 1 -- 172.21.15.69:0/1072729401 learned_addr learned my addr 172.21.15.69:0/1072729401 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:12.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.887+0000 7f0e46556640 1 -- 172.21.15.69:0/1072729401 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e401644e0 msgr2=0x7f0e401648f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:12.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.887+0000 7f0e45d55640 1 --2- 172.21.15.69:0/1072729401 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e401644e0 0x7f0e401648f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:12.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.887+0000 7f0e46556640 1 --2- 172.21.15.69:0/1072729401 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e401644e0 0x7f0e401648f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:12.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.887+0000 7f0e46556640 1 -- 172.21.15.69:0/1072729401 >> v1:172.21.15.146:6789/0 conn(0x7f0e400d1f30 legacy=0x7f0e400d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:12.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.887+0000 7f0e46556640 1 -- 172.21.15.69:0/1072729401 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e401608b0 con 0x7f0e400b8ca0 2024-04-25T10:59:12.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.887+0000 7f0e46556640 1 --2- 172.21.15.69:0/1072729401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e400b8ca0 0x7f0e40163fa0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c009870 tx=0x7f0e3c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787c315f612be7f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:12.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.887+0000 7f0e45554640 1 -- 172.21.15.69:0/1072729401 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c037070 con 0x7f0e400b8ca0 2024-04-25T10:59:12.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.888+0000 7f0e45554640 1 -- 172.21.15.69:0/1072729401 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e3c032440 con 0x7f0e400b8ca0 2024-04-25T10:59:12.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.888+0000 7f0e45554640 1 -- 172.21.15.69:0/1072729401 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c032810 con 0x7f0e400b8ca0 2024-04-25T10:59:12.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.888+0000 7f0e487ba640 1 -- 172.21.15.69:0/1072729401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e400b8ca0 msgr2=0x7f0e40163fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:12.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.888+0000 7f0e487ba640 1 --2- 172.21.15.69:0/1072729401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e400b8ca0 0x7f0e40163fa0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c009870 tx=0x7f0e3c02a210 comp rx=0 tx=0).stop 2024-04-25T10:59:12.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.888+0000 7f0e487ba640 1 -- 172.21.15.69:0/1072729401 shutdown_connections 2024-04-25T10:59:12.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.888+0000 7f0e487ba640 1 --2- 172.21.15.69:0/1072729401 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e401644e0 0x7f0e401648f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:12.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.888+0000 7f0e487ba640 1 --2- 172.21.15.69:0/1072729401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e400b8ca0 0x7f0e40163fa0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:12.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.888+0000 7f0e487ba640 1 -- 172.21.15.69:0/1072729401 >> 172.21.15.69:0/1072729401 conn(0x7f0e400d3aa0 msgr2=0x7f0e400d3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:12.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.888+0000 7f0e487ba640 1 -- 172.21.15.69:0/1072729401 shutdown_connections 2024-04-25T10:59:12.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.888+0000 7f0e487ba640 1 -- 172.21.15.69:0/1072729401 wait complete. 2024-04-25T10:59:12.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.890+0000 7f0e487ba640 1 Processor -- start 2024-04-25T10:59:12.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.890+0000 7f0e487ba640 1 -- start start 2024-04-25T10:59:12.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.891+0000 7f0e487ba640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e400b8ca0 0x7f0e401ea7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:12.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.891+0000 7f0e487ba640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e400d1f30 0x7f0e40071a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:12.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.891+0000 7f0e487ba640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e401644e0 0x7f0e400cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:12.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.891+0000 7f0e487ba640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4006e280 con 0x7f0e401644e0 2024-04-25T10:59:12.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.891+0000 7f0e487ba640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0e400d32c0 con 0x7f0e400b8ca0 2024-04-25T10:59:12.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.891+0000 7f0e487ba640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0e4006e560 con 0x7f0e400d1f30 2024-04-25T10:59:12.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.891+0000 7f0e46d57640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e401644e0 0x7f0e400cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:12.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.891+0000 7f0e46d57640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e401644e0 0x7f0e400cb6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55254/0 (socket says 172.21.15.69:55254) 2024-04-25T10:59:12.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.892+0000 7f0e46d57640 1 -- 172.21.15.69:0/3968164976 learned_addr learned my addr 172.21.15.69:0/3968164976 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:12.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.892+0000 7f0e45d55640 1 --2- 172.21.15.69:0/3968164976 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e400d1f30 0x7f0e40071a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:12.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.892+0000 7f0e46556640 1 --2- 172.21.15.69:0/3968164976 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e400b8ca0 0x7f0e401ea7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:12.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.892+0000 7f0e46d57640 1 -- 172.21.15.69:0/3968164976 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e400d1f30 msgr2=0x7f0e40071a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:12.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.892+0000 7f0e46d57640 1 --2- 172.21.15.69:0/3968164976 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e400d1f30 0x7f0e40071a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:12.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.892+0000 7f0e46d57640 1 -- 172.21.15.69:0/3968164976 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e400b8ca0 msgr2=0x7f0e401ea7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:12.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.892+0000 7f0e46d57640 1 --2- 172.21.15.69:0/3968164976 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e400b8ca0 0x7f0e401ea7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:12.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.892+0000 7f0e46d57640 1 -- 172.21.15.69:0/3968164976 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e4015e540 con 0x7f0e401644e0 2024-04-25T10:59:12.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.892+0000 7f0e46d57640 1 --2- 172.21.15.69:0/3968164976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e401644e0 0x7f0e400cb6f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0e3800b9a0 tx=0x7f0e3800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:12.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.892+0000 7f0e36ffd640 1 -- 172.21.15.69:0/3968164976 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e38019070 con 0x7f0e401644e0 2024-04-25T10:59:12.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.892+0000 7f0e36ffd640 1 -- 172.21.15.69:0/3968164976 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e3800cba0 con 0x7f0e401644e0 2024-04-25T10:59:12.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.892+0000 7f0e487ba640 1 -- 172.21.15.69:0/3968164976 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e4014bbb0 con 0x7f0e401644e0 2024-04-25T10:59:12.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.892+0000 7f0e36ffd640 1 -- 172.21.15.69:0/3968164976 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e38014440 con 0x7f0e401644e0 2024-04-25T10:59:12.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.892+0000 7f0e487ba640 1 -- 172.21.15.69:0/3968164976 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e400cac10 con 0x7f0e401644e0 2024-04-25T10:59:12.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.893+0000 7f0e487ba640 1 -- 172.21.15.69:0/3968164976 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e4014bd80 con 0x7f0e401644e0 2024-04-25T10:59:12.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.894+0000 7f0e36ffd640 1 -- 172.21.15.69:0/3968164976 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0e3801c070 con 0x7f0e401644e0 2024-04-25T10:59:12.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.894+0000 7f0e36ffd640 1 --2- 172.21.15.69:0/3968164976 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0e240744b0 0x7f0e240769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:12.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.894+0000 7f0e46556640 1 --2- 172.21.15.69:0/3968164976 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0e240744b0 0x7f0e240769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:12.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.894+0000 7f0e36ffd640 1 -- 172.21.15.69:0/3968164976 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f0e38094950 con 0x7f0e401644e0 2024-04-25T10:59:12.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.895+0000 7f0e46556640 1 --2- 172.21.15.69:0/3968164976 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0e240744b0 0x7f0e240769c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c034000 tx=0x7f0e3c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:12.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:12.899+0000 7f0e36ffd640 1 -- 172.21.15.69:0/3968164976 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0e4015e540 con 0x7f0e401644e0 2024-04-25T10:59:13.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:13.110+0000 7f0e487ba640 1 -- 172.21.15.69:0/3968164976 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e40050740 con 0x7f0e401644e0 2024-04-25T10:59:13.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:13.110+0000 7f0e36ffd640 1 -- 172.21.15.69:0/3968164976 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7f0e3805ef00 con 0x7f0e401644e0 2024-04-25T10:59:13.111 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:13.111 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1848276963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1848276963},{"type":"v1","addr":"172.21.15.69:6837","nonce":1848276963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T10:59:13.111 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 2 2024-04-25T10:59:13.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:13.112+0000 7f0e487ba640 1 -- 172.21.15.69:0/3968164976 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0e240744b0 msgr2=0x7f0e240769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:13.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:13.112+0000 7f0e487ba640 1 --2- 172.21.15.69:0/3968164976 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0e240744b0 0x7f0e240769c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c034000 tx=0x7f0e3c02e000 comp rx=0 tx=0).stop 2024-04-25T10:59:13.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:13.112+0000 7f0e487ba640 1 -- 172.21.15.69:0/3968164976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e401644e0 msgr2=0x7f0e400cb6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:13.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:13.112+0000 7f0e487ba640 1 --2- 172.21.15.69:0/3968164976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e401644e0 0x7f0e400cb6f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0e3800b9a0 tx=0x7f0e3800be60 comp rx=0 tx=0).stop 2024-04-25T10:59:13.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:13.112+0000 7f0e487ba640 1 -- 172.21.15.69:0/3968164976 shutdown_connections 2024-04-25T10:59:13.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:13.112+0000 7f0e487ba640 1 --2- 172.21.15.69:0/3968164976 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0e240744b0 0x7f0e240769c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:13.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:13.112+0000 7f0e487ba640 1 --2- 172.21.15.69:0/3968164976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e401644e0 0x7f0e400cb6f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:13.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:13.112+0000 7f0e487ba640 1 --2- 172.21.15.69:0/3968164976 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e400d1f30 0x7f0e40071a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:13.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:13.112+0000 7f0e487ba640 1 --2- 172.21.15.69:0/3968164976 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e400b8ca0 0x7f0e401ea7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:13.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:13.112+0000 7f0e487ba640 1 -- 172.21.15.69:0/3968164976 >> 172.21.15.69:0/3968164976 conn(0x7f0e400d3aa0 msgr2=0x7f0e400b68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:13.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:13.112+0000 7f0e487ba640 1 -- 172.21.15.69:0/3968164976 shutdown_connections 2024-04-25T10:59:13.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:13.112+0000 7f0e487ba640 1 -- 172.21.15.69:0/3968164976 wait complete. 2024-04-25T10:59:15.125 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:59:15.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.245+0000 7f01c71f3640 1 Processor -- start 2024-04-25T10:59:15.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.245+0000 7f01c71f3640 1 -- start start 2024-04-25T10:59:15.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.247+0000 7f01c71f3640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f01c00b6a00 0x7f01c015d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:15.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.247+0000 7f01c71f3640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01c015a2f0 0x7f01c006ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:15.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.247+0000 7f01c71f3640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01c01611e0 con 0x7f01c006d390 2024-04-25T10:59:15.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.247+0000 7f01c71f3640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f01c0160d50 con 0x7f01c00b6a00 2024-04-25T10:59:15.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.247+0000 7f01c71f3640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f01c015a700 con 0x7f01c015a2f0 2024-04-25T10:59:15.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.247+0000 7f01c4f8f640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f01c00b6a00 0x7f01c015d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:15.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.247+0000 7f01c4f8f640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f01c00b6a00 0x7f01c015d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55270/0 (socket says 172.21.15.69:55270) 2024-04-25T10:59:15.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.247+0000 7f01c4f8f640 1 -- 172.21.15.69:0/2423508978 learned_addr learned my addr 172.21.15.69:0/2423508978 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:15.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.247+0000 7f01c5790640 1 --1- 172.21.15.69:0/2423508978 >> v1:172.21.15.69:6789/0 conn(0x7f01c006d390 0x7f01c006f090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:33690/0 (socket says 172.21.15.69:33690) 2024-04-25T10:59:15.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.247+0000 7f01bffff640 1 --2- 172.21.15.69:0/2423508978 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01c015a2f0 0x7f01c006ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:15.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.247+0000 7f01c4f8f640 1 -- 172.21.15.69:0/2423508978 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01c015a2f0 msgr2=0x7f01c006ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:15.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.247+0000 7f01c4f8f640 1 --2- 172.21.15.69:0/2423508978 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01c015a2f0 0x7f01c006ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:15.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.247+0000 7f01c4f8f640 1 -- 172.21.15.69:0/2423508978 >> v1:172.21.15.69:6789/0 conn(0x7f01c006d390 legacy=0x7f01c006f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:15.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.247+0000 7f01c4f8f640 1 -- 172.21.15.69:0/2423508978 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c006d7a0 con 0x7f01c00b6a00 2024-04-25T10:59:15.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.247+0000 7f01bffff640 1 --2- 172.21.15.69:0/2423508978 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01c015a2f0 0x7f01c006ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:15.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.248+0000 7f01c4f8f640 1 --2- 172.21.15.69:0/2423508978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01c00b6a00 0x7f01c015d6c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f01b0009870 tx=0x7f01b002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930ed4b39923b8c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:15.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.248+0000 7f01bf7fe640 1 -- 172.21.15.69:0/2423508978 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b0037070 con 0x7f01c00b6a00 2024-04-25T10:59:15.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.248+0000 7f01bf7fe640 1 -- 172.21.15.69:0/2423508978 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01b0032440 con 0x7f01c00b6a00 2024-04-25T10:59:15.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.248+0000 7f01bf7fe640 1 -- 172.21.15.69:0/2423508978 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b0032810 con 0x7f01c00b6a00 2024-04-25T10:59:15.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.249+0000 7f01c71f3640 1 -- 172.21.15.69:0/2423508978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01c00b6a00 msgr2=0x7f01c015d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:15.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.249+0000 7f01c71f3640 1 --2- 172.21.15.69:0/2423508978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01c00b6a00 0x7f01c015d6c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f01b0009870 tx=0x7f01b002a250 comp rx=0 tx=0).stop 2024-04-25T10:59:15.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.249+0000 7f01c71f3640 1 -- 172.21.15.69:0/2423508978 shutdown_connections 2024-04-25T10:59:15.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.249+0000 7f01c71f3640 1 --2- 172.21.15.69:0/2423508978 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01c015a2f0 0x7f01c006ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:15.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.249+0000 7f01c71f3640 1 --2- 172.21.15.69:0/2423508978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01c00b6a00 0x7f01c015d6c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:15.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.249+0000 7f01c71f3640 1 -- 172.21.15.69:0/2423508978 >> 172.21.15.69:0/2423508978 conn(0x7f01c00d1ae0 msgr2=0x7f01c00d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:15.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.249+0000 7f01c71f3640 1 -- 172.21.15.69:0/2423508978 shutdown_connections 2024-04-25T10:59:15.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.249+0000 7f01c71f3640 1 -- 172.21.15.69:0/2423508978 wait complete. 2024-04-25T10:59:15.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.250+0000 7f01c71f3640 1 Processor -- start 2024-04-25T10:59:15.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.250+0000 7f01c71f3640 1 -- start start 2024-04-25T10:59:15.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.252+0000 7f01c71f3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01c006d390 0x7f01c014c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:15.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.252+0000 7f01c71f3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01c00b6a00 0x7f01c00cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:15.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.252+0000 7f01c71f3640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01c015a2f0 0x7f01c0073260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:15.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.252+0000 7f01c71f3640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01609e0 con 0x7f01c006d390 2024-04-25T10:59:15.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.252+0000 7f01c71f3640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f01c0160700 con 0x7f01c00b6a00 2024-04-25T10:59:15.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.252+0000 7f01c71f3640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f01c00b5980 con 0x7f01c015a2f0 2024-04-25T10:59:15.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.252+0000 7f01c4f8f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01c006d390 0x7f01c014c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:15.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.252+0000 7f01c4f8f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01c006d390 0x7f01c014c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55278/0 (socket says 172.21.15.69:55278) 2024-04-25T10:59:15.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.252+0000 7f01c4f8f640 1 -- 172.21.15.69:0/1927215676 learned_addr learned my addr 172.21.15.69:0/1927215676 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:15.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.252+0000 7f01c5790640 1 --2- 172.21.15.69:0/1927215676 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01c015a2f0 0x7f01c0073260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:15.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.252+0000 7f01bffff640 1 --2- 172.21.15.69:0/1927215676 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01c00b6a00 0x7f01c00cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:15.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.252+0000 7f01c4f8f640 1 -- 172.21.15.69:0/1927215676 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01c015a2f0 msgr2=0x7f01c0073260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:15.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.252+0000 7f01c4f8f640 1 --2- 172.21.15.69:0/1927215676 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01c015a2f0 0x7f01c0073260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:15.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.252+0000 7f01c4f8f640 1 -- 172.21.15.69:0/1927215676 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01c00b6a00 msgr2=0x7f01c00cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:15.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.252+0000 7f01c4f8f640 1 --2- 172.21.15.69:0/1927215676 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01c00b6a00 0x7f01c00cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:15.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.252+0000 7f01c4f8f640 1 -- 172.21.15.69:0/1927215676 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c01575f0 con 0x7f01c006d390 2024-04-25T10:59:15.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.252+0000 7f01bffff640 1 --2- 172.21.15.69:0/1927215676 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01c00b6a00 0x7f01c00cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:15.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.252+0000 7f01c4f8f640 1 --2- 172.21.15.69:0/1927215676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01c006d390 0x7f01c014c340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f01b0000c00 tx=0x7f01b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:15.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.253+0000 7f01bd7fa640 1 -- 172.21.15.69:0/1927215676 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b0037040 con 0x7f01c006d390 2024-04-25T10:59:15.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.253+0000 7f01bd7fa640 1 -- 172.21.15.69:0/1927215676 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01b0042070 con 0x7f01c006d390 2024-04-25T10:59:15.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.253+0000 7f01c71f3640 1 -- 172.21.15.69:0/1927215676 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c01578d0 con 0x7f01c006d390 2024-04-25T10:59:15.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.253+0000 7f01bd7fa640 1 -- 172.21.15.69:0/1927215676 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b0030bd0 con 0x7f01c006d390 2024-04-25T10:59:15.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.253+0000 7f01c71f3640 1 -- 172.21.15.69:0/1927215676 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c0157e90 con 0x7f01c006d390 2024-04-25T10:59:15.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.254+0000 7f01c71f3640 1 -- 172.21.15.69:0/1927215676 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01c00cf750 con 0x7f01c006d390 2024-04-25T10:59:15.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.254+0000 7f01bd7fa640 1 -- 172.21.15.69:0/1927215676 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f01b0047460 con 0x7f01c006d390 2024-04-25T10:59:15.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.255+0000 7f01bd7fa640 1 --2- 172.21.15.69:0/1927215676 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f01ac0744b0 0x7f01ac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:15.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.255+0000 7f01bffff640 1 --2- 172.21.15.69:0/1927215676 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f01ac0744b0 0x7f01ac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:15.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.255+0000 7f01bd7fa640 1 -- 172.21.15.69:0/1927215676 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f01b00b5a40 con 0x7f01c006d390 2024-04-25T10:59:15.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.255+0000 7f01bffff640 1 --2- 172.21.15.69:0/1927215676 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f01ac0744b0 0x7f01ac0769c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f01c00b2390 tx=0x7f01b4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:15.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.259+0000 7f01bd7fa640 1 -- 172.21.15.69:0/1927215676 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f01c00cf750 con 0x7f01c006d390 2024-04-25T10:59:15.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.470+0000 7f01c71f3640 1 -- 172.21.15.69:0/1927215676 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01c00d1ef0 con 0x7f01c006d390 2024-04-25T10:59:15.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.471+0000 7f01bd7fa640 1 -- 172.21.15.69:0/1927215676 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3867 (secure 0 0 0) 0x7f01b00b5e20 con 0x7f01c006d390 2024-04-25T10:59:15.473 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:15.474 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/2361834023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2361834023},{"type":"v1","addr":"172.21.15.146:6833","nonce":2361834023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1848276963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1848276963},{"type":"v1","addr":"172.21.15.69:6837","nonce":1848276963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1826742846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1826742846},{"type":"v1","addr":"172.21.15.69:6836","nonce":1826742846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/908008797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":908008797},{"type":"v1","addr":"172.21.15.146:6835","nonce":908008797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T10:59:15.475 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 3 2024-04-25T10:59:15.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.473+0000 7f01c71f3640 1 -- 172.21.15.69:0/1927215676 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f01ac0744b0 msgr2=0x7f01ac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:15.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.473+0000 7f01c71f3640 1 --2- 172.21.15.69:0/1927215676 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f01ac0744b0 0x7f01ac0769c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f01c00b2390 tx=0x7f01b4008040 comp rx=0 tx=0).stop 2024-04-25T10:59:15.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.473+0000 7f01c71f3640 1 -- 172.21.15.69:0/1927215676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01c006d390 msgr2=0x7f01c014c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:15.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.473+0000 7f01c71f3640 1 --2- 172.21.15.69:0/1927215676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01c006d390 0x7f01c014c340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f01b0000c00 tx=0x7f01b00046c0 comp rx=0 tx=0).stop 2024-04-25T10:59:15.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.473+0000 7f01c71f3640 1 -- 172.21.15.69:0/1927215676 shutdown_connections 2024-04-25T10:59:15.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.473+0000 7f01c71f3640 1 --2- 172.21.15.69:0/1927215676 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f01ac0744b0 0x7f01ac0769c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:15.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.473+0000 7f01c71f3640 1 --2- 172.21.15.69:0/1927215676 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f01c015a2f0 0x7f01c0073260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:15.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.473+0000 7f01c71f3640 1 --2- 172.21.15.69:0/1927215676 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01c00b6a00 0x7f01c00cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:15.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.473+0000 7f01c71f3640 1 --2- 172.21.15.69:0/1927215676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01c006d390 0x7f01c014c340 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:15.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.473+0000 7f01c71f3640 1 -- 172.21.15.69:0/1927215676 >> 172.21.15.69:0/1927215676 conn(0x7f01c00d1ae0 msgr2=0x7f01c00cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:15.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.473+0000 7f01c71f3640 1 -- 172.21.15.69:0/1927215676 shutdown_connections 2024-04-25T10:59:15.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.473+0000 7f01c71f3640 1 -- 172.21.15.69:0/1927215676 wait complete. 2024-04-25T10:59:15.485 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:59:15.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.592+0000 7f1c642e7640 1 Processor -- start 2024-04-25T10:59:15.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.592+0000 7f1c642e7640 1 -- start start 2024-04-25T10:59:15.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.594+0000 7f1c642e7640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f1c5c06ccd0 0x7f1c5c0d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:15.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.594+0000 7f1c642e7640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c5c0d1850 con 0x7f1c5c0d4a50 2024-04-25T10:59:15.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.594+0000 7f1c642e7640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c5c160de0 con 0x7f1c5c0d0e00 2024-04-25T10:59:15.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.594+0000 7f1c642e7640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f1c5c1610c0 con 0x7f1c5c06ccd0 2024-04-25T10:59:15.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.594+0000 7f1c61882640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f1c5c06ccd0 0x7f1c5c0d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:15.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.594+0000 7f1c62884640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f1c5c0d4a50 0x7f1c5c0d4e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:57062/0 (socket says 172.21.15.69:57062) 2024-04-25T10:59:15.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.594+0000 7f1c61882640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f1c5c06ccd0 0x7f1c5c0d1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40992/0 (socket says 172.21.15.69:40992) 2024-04-25T10:59:15.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.594+0000 7f1c62884640 1 -- 172.21.15.69:0/2282116946 learned_addr learned my addr 172.21.15.69:0/2282116946 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:15.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.594+0000 7f1c61882640 1 -- 172.21.15.69:0/2282116946 >> v1:172.21.15.146:6789/0 conn(0x7f1c5c0d0e00 legacy=0x7f1c5c06c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:15.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.594+0000 7f1c61882640 1 -- 172.21.15.69:0/2282116946 >> v1:172.21.15.69:6789/0 conn(0x7f1c5c0d4a50 legacy=0x7f1c5c0d4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:15.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.594+0000 7f1c61882640 1 -- 172.21.15.69:0/2282116946 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c5c1613a0 con 0x7f1c5c06ccd0 2024-04-25T10:59:15.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.595+0000 7f1c61882640 1 --2- 172.21.15.69:0/2282116946 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c5c06ccd0 0x7f1c5c0d1210 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c0097a0 tx=0x7f1c4c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6606a7d965d2409f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:15.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.598+0000 7f1c61081640 1 -- 172.21.15.69:0/2282116946 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c037070 con 0x7f1c5c06ccd0 2024-04-25T10:59:15.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.598+0000 7f1c61081640 1 -- 172.21.15.69:0/2282116946 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c4c032440 con 0x7f1c5c06ccd0 2024-04-25T10:59:15.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.598+0000 7f1c61081640 1 -- 172.21.15.69:0/2282116946 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c032810 con 0x7f1c5c06ccd0 2024-04-25T10:59:15.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.598+0000 7f1c642e7640 1 -- 172.21.15.69:0/2282116946 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c5c06ccd0 msgr2=0x7f1c5c0d1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:15.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.598+0000 7f1c642e7640 1 --2- 172.21.15.69:0/2282116946 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c5c06ccd0 0x7f1c5c0d1210 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c0097a0 tx=0x7f1c4c02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:59:15.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.599+0000 7f1c642e7640 1 -- 172.21.15.69:0/2282116946 shutdown_connections 2024-04-25T10:59:15.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.599+0000 7f1c642e7640 1 --2- 172.21.15.69:0/2282116946 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c5c06ccd0 0x7f1c5c0d1210 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:15.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.599+0000 7f1c642e7640 1 -- 172.21.15.69:0/2282116946 >> 172.21.15.69:0/2282116946 conn(0x7f1c5c0d5d90 msgr2=0x7f1c5c0d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:15.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.599+0000 7f1c642e7640 1 -- 172.21.15.69:0/2282116946 shutdown_connections 2024-04-25T10:59:15.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.599+0000 7f1c642e7640 1 -- 172.21.15.69:0/2282116946 wait complete. 2024-04-25T10:59:15.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.600+0000 7f1c642e7640 1 Processor -- start 2024-04-25T10:59:15.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.600+0000 7f1c642e7640 1 -- start start 2024-04-25T10:59:15.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.602+0000 7f1c642e7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c5c0d0e00 0x7f1c5c1582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:15.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.602+0000 7f1c642e7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c5c0d4a50 0x7f1c5c0c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:15.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.602+0000 7f1c642e7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c5c0cf160 0x7f1c5c1ea890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:15.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.602+0000 7f1c642e7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c164e00 con 0x7f1c5c0d0e00 2024-04-25T10:59:15.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.602+0000 7f1c642e7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c0b5320 con 0x7f1c5c0d4a50 2024-04-25T10:59:15.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.602+0000 7f1c642e7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1c5c164f80 con 0x7f1c5c0cf160 2024-04-25T10:59:15.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.602+0000 7f1c62083640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c5c0d0e00 0x7f1c5c1582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:15.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.602+0000 7f1c62083640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c5c0d0e00 0x7f1c5c1582c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41002/0 (socket says 172.21.15.69:41002) 2024-04-25T10:59:15.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.602+0000 7f1c62083640 1 -- 172.21.15.69:0/3726836143 learned_addr learned my addr 172.21.15.69:0/3726836143 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:15.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.602+0000 7f1c61882640 1 --2- 172.21.15.69:0/3726836143 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c5c0d4a50 0x7f1c5c0c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:15.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.602+0000 7f1c62884640 1 --2- 172.21.15.69:0/3726836143 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c5c0cf160 0x7f1c5c1ea890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:15.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.602+0000 7f1c62083640 1 -- 172.21.15.69:0/3726836143 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c5c0cf160 msgr2=0x7f1c5c1ea890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:15.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.602+0000 7f1c62083640 1 --2- 172.21.15.69:0/3726836143 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c5c0cf160 0x7f1c5c1ea890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:15.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.602+0000 7f1c62083640 1 -- 172.21.15.69:0/3726836143 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c5c0d4a50 msgr2=0x7f1c5c0c98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:15.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.602+0000 7f1c62083640 1 --2- 172.21.15.69:0/3726836143 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c5c0d4a50 0x7f1c5c0c98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:15.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.602+0000 7f1c62083640 1 -- 172.21.15.69:0/3726836143 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c5c0cea20 con 0x7f1c5c0d0e00 2024-04-25T10:59:15.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.602+0000 7f1c62884640 1 --2- 172.21.15.69:0/3726836143 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c5c0cf160 0x7f1c5c1ea890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:15.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.602+0000 7f1c62083640 1 --2- 172.21.15.69:0/3726836143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c5c0d0e00 0x7f1c5c1582c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800c3f0 tx=0x7f1c5800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:15.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.602+0000 7f1c52ffd640 1 -- 172.21.15.69:0/3726836143 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c580052d0 con 0x7f1c5c0d0e00 2024-04-25T10:59:15.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.602+0000 7f1c52ffd640 1 -- 172.21.15.69:0/3726836143 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c58005490 con 0x7f1c5c0d0e00 2024-04-25T10:59:15.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.602+0000 7f1c52ffd640 1 -- 172.21.15.69:0/3726836143 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c58007260 con 0x7f1c5c0d0e00 2024-04-25T10:59:15.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.602+0000 7f1c642e7640 1 -- 172.21.15.69:0/3726836143 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c5c0ced00 con 0x7f1c5c0d0e00 2024-04-25T10:59:15.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.602+0000 7f1c642e7640 1 -- 172.21.15.69:0/3726836143 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c5c1ea370 con 0x7f1c5c0d0e00 2024-04-25T10:59:15.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.603+0000 7f1c642e7640 1 -- 172.21.15.69:0/3726836143 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c5c050740 con 0x7f1c5c0d0e00 2024-04-25T10:59:15.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.604+0000 7f1c52ffd640 1 -- 172.21.15.69:0/3726836143 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1c58007400 con 0x7f1c5c0d0e00 2024-04-25T10:59:15.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.604+0000 7f1c52ffd640 1 --2- 172.21.15.69:0/3726836143 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1c400744e0 0x7f1c400769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:15.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.605+0000 7f1c52ffd640 1 -- 172.21.15.69:0/3726836143 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f1c58094370 con 0x7f1c5c0d0e00 2024-04-25T10:59:15.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.605+0000 7f1c61882640 1 --2- 172.21.15.69:0/3726836143 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1c400744e0 0x7f1c400769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:15.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.605+0000 7f1c61882640 1 --2- 172.21.15.69:0/3726836143 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1c400744e0 0x7f1c400769f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c034000 tx=0x7f1c4c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:15.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.609+0000 7f1c52ffd640 1 -- 172.21.15.69:0/3726836143 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1c5805ee50 con 0x7f1c5c0d0e00 2024-04-25T10:59:15.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.783+0000 7f1c642e7640 1 -- 172.21.15.69:0/3726836143 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1c5c15e940 con 0x7f1c5c0d0e00 2024-04-25T10:59:15.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.784+0000 7f1c52ffd640 1 -- 172.21.15.69:0/3726836143 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12884 (secure 0 0 0) 0x7f1c5805e750 con 0x7f1c5c0d0e00 2024-04-25T10:59:15.784 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:15.785 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":16,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T10:59:08.565477+0000","last_up_change":"2024-04-25T10:59:05.546839+0000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:59:06.422674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6825","nonce":1876101065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6827","nonce":1876101065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6831","nonce":1876101065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6829","nonce":1876101065}]},"public_addr":"172.21.15.69:6825/1876101065","cluster_addr":"172.21.15.69:6827/1876101065","heartbeat_back_addr":"172.21.15.69:6831/1876101065","heartbeat_front_addr":"172.21.15.69:6829/1876101065","state":["exists","up"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6801","nonce":1806438563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6803","nonce":1806438563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6807","nonce":1806438563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6805","nonce":1806438563}]},"public_addr":"172.21.15.69:6801/1806438563","cluster_addr":"172.21.15.69:6803/1806438563","heartbeat_back_addr":"172.21.15.69:6807/1806438563","heartbeat_front_addr":"172.21.15.69:6805/1806438563","state":["exists","up"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6817","nonce":1593617983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6819","nonce":1593617983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6823","nonce":1593617983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6821","nonce":1593617983}]},"public_addr":"172.21.15.69:6817/1593617983","cluster_addr":"172.21.15.69:6819/1593617983","heartbeat_back_addr":"172.21.15.69:6823/1593617983","heartbeat_front_addr":"172.21.15.69:6821/1593617983","state":["exists","up"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6809","nonce":3950661294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6811","nonce":3950661294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6815","nonce":3950661294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6813","nonce":3950661294}]},"public_addr":"172.21.15.69:6809/3950661294","cluster_addr":"172.21.15.69:6811/3950661294","heartbeat_back_addr":"172.21.15.69:6815/3950661294","heartbeat_front_addr":"172.21.15.69:6813/3950661294","state":["exists","up"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6801","nonce":2508929872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6803","nonce":2508929872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6807","nonce":2508929872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6805","nonce":2508929872}]},"public_addr":"172.21.15.146:6801/2508929872","cluster_addr":"172.21.15.146:6803/2508929872","heartbeat_back_addr":"172.21.15.146:6807/2508929872","heartbeat_front_addr":"172.21.15.146:6805/2508929872","state":["exists","up"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6811","nonce":1310911138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6816","nonce":1310911138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6823","nonce":1310911138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6821","nonce":1310911138}]},"public_addr":"172.21.15.146:6811/1310911138","cluster_addr":"172.21.15.146:6816/1310911138","heartbeat_back_addr":"172.21.15.146:6823/1310911138","heartbeat_front_addr":"172.21.15.146:6821/1310911138","state":["exists","up"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6810","nonce":634688021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6813","nonce":634688021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6820","nonce":634688021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6817","nonce":634688021}]},"public_addr":"172.21.15.146:6810/634688021","cluster_addr":"172.21.15.146:6813/634688021","heartbeat_back_addr":"172.21.15.146:6820/634688021","heartbeat_front_addr":"172.21.15.146:6817/634688021","state":["exists","up"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6825","nonce":3814215935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6827","nonce":3814215935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6831","nonce":3814215935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6829","nonce":3814215935}]},"public_addr":"172.21.15.146:6825/3814215935","cluster_addr":"172.21.15.146:6827/3814215935","heartbeat_back_addr":"172.21.15.146:6831/3814215935","heartbeat_front_addr":"172.21.15.146:6829/3814215935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.820200+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.850189+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.764560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.866248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.807897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.869881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.782310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.827131+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:59:15.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.786+0000 7f1c642e7640 1 -- 172.21.15.69:0/3726836143 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1c400744e0 msgr2=0x7f1c400769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:15.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.786+0000 7f1c642e7640 1 --2- 172.21.15.69:0/3726836143 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1c400744e0 0x7f1c400769f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c034000 tx=0x7f1c4c02e000 comp rx=0 tx=0).stop 2024-04-25T10:59:15.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.786+0000 7f1c642e7640 1 -- 172.21.15.69:0/3726836143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c5c0d0e00 msgr2=0x7f1c5c1582c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:15.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.786+0000 7f1c642e7640 1 --2- 172.21.15.69:0/3726836143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c5c0d0e00 0x7f1c5c1582c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800c3f0 tx=0x7f1c5800c8b0 comp rx=0 tx=0).stop 2024-04-25T10:59:15.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.786+0000 7f1c642e7640 1 -- 172.21.15.69:0/3726836143 shutdown_connections 2024-04-25T10:59:15.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.786+0000 7f1c642e7640 1 --2- 172.21.15.69:0/3726836143 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1c400744e0 0x7f1c400769f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:15.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.786+0000 7f1c642e7640 1 --2- 172.21.15.69:0/3726836143 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1c5c0cf160 0x7f1c5c1ea890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:15.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.786+0000 7f1c642e7640 1 --2- 172.21.15.69:0/3726836143 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c5c0d4a50 0x7f1c5c0c98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:15.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.786+0000 7f1c642e7640 1 --2- 172.21.15.69:0/3726836143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c5c0d0e00 0x7f1c5c1582c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:15.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.786+0000 7f1c642e7640 1 -- 172.21.15.69:0/3726836143 >> 172.21.15.69:0/3726836143 conn(0x7f1c5c0d5d90 msgr2=0x7f1c5c163880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:15.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.786+0000 7f1c642e7640 1 -- 172.21.15.69:0/3726836143 shutdown_connections 2024-04-25T10:59:15.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.786+0000 7f1c642e7640 1 -- 172.21.15.69:0/3726836143 wait complete. 2024-04-25T10:59:15.797 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:59:06.422674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:59:15.798 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:59:15.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.908+0000 7f86e31f3640 1 Processor -- start 2024-04-25T10:59:15.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.908+0000 7f86e31f3640 1 -- start start 2024-04-25T10:59:15.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.909+0000 7f86e31f3640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f86dc0cec90 0x7f86dc0d2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:15.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.909+0000 7f86e31f3640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86dc0ce5d0 con 0x7f86dc0ce1c0 2024-04-25T10:59:15.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.909+0000 7f86e31f3640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86dc0cf3c0 con 0x7f86dc0b3f20 2024-04-25T10:59:15.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.909+0000 7f86e31f3640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f86dc0cf6a0 con 0x7f86dc0cec90 2024-04-25T10:59:15.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.909+0000 7f86e0f8f640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f86dc0ce1c0 0x7f86dc0d2580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:57074/0 (socket says 172.21.15.69:57074) 2024-04-25T10:59:15.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.909+0000 7f86e0f8f640 1 -- 172.21.15.69:0/3779566404 learned_addr learned my addr 172.21.15.69:0/3779566404 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:15.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.909+0000 7f86dbfff640 1 --2- 172.21.15.69:0/3779566404 >> v2:172.21.15.69:3300/0 conn(0x7f86dc0cec90 0x7f86dc0d2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:15.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.909+0000 7f86dbfff640 1 -- 172.21.15.69:0/3779566404 >> v1:172.21.15.146:6789/0 conn(0x7f86dc0b3f20 legacy=0x7f86dc0b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:15.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.909+0000 7f86dbfff640 1 -- 172.21.15.69:0/3779566404 >> v1:172.21.15.69:6789/0 conn(0x7f86dc0ce1c0 legacy=0x7f86dc0d2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:15.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.909+0000 7f86dbfff640 1 -- 172.21.15.69:0/3779566404 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86dc0cf980 con 0x7f86dc0cec90 2024-04-25T10:59:15.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.909+0000 7f86db7fe640 1 -- 172.21.15.69:0/3779566404 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3448982118 0 0) 0x7f86dc0ce5d0 con 0x7f86dc0ce1c0 2024-04-25T10:59:15.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.910+0000 7f86dbfff640 1 --2- 172.21.15.69:0/3779566404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86dc0cec90 0x7f86dc0d2c90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0097a0 tx=0x7f86cc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62451a4dc470eff2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:15.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.910+0000 7f86db7fe640 1 -- 172.21.15.69:0/3779566404 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc037070 con 0x7f86dc0cec90 2024-04-25T10:59:15.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.910+0000 7f86db7fe640 1 -- 172.21.15.69:0/3779566404 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86cc02ae50 con 0x7f86dc0cec90 2024-04-25T10:59:15.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.910+0000 7f86db7fe640 1 -- 172.21.15.69:0/3779566404 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc032660 con 0x7f86dc0cec90 2024-04-25T10:59:15.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.911+0000 7f86e31f3640 1 -- 172.21.15.69:0/3779566404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86dc0cec90 msgr2=0x7f86dc0d2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:15.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.911+0000 7f86e31f3640 1 --2- 172.21.15.69:0/3779566404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86dc0cec90 0x7f86dc0d2c90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0097a0 tx=0x7f86cc02a140 comp rx=0 tx=0).stop 2024-04-25T10:59:15.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.911+0000 7f86e31f3640 1 -- 172.21.15.69:0/3779566404 shutdown_connections 2024-04-25T10:59:15.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.911+0000 7f86e31f3640 1 --2- 172.21.15.69:0/3779566404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86dc0cec90 0x7f86dc0d2c90 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:15.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.911+0000 7f86e31f3640 1 -- 172.21.15.69:0/3779566404 >> 172.21.15.69:0/3779566404 conn(0x7f86dc09b7d0 msgr2=0x7f86dc09bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:15.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.911+0000 7f86e31f3640 1 -- 172.21.15.69:0/3779566404 shutdown_connections 2024-04-25T10:59:15.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.911+0000 7f86e31f3640 1 -- 172.21.15.69:0/3779566404 wait complete. 2024-04-25T10:59:15.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.913+0000 7f86e31f3640 1 Processor -- start 2024-04-25T10:59:15.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.913+0000 7f86e31f3640 1 -- start start 2024-04-25T10:59:15.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.914+0000 7f86e31f3640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86dc0b3f20 0x7f86dc151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:15.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.914+0000 7f86e31f3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86dc0ce1c0 0x7f86dc162b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:15.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.914+0000 7f86e31f3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86dc0cec90 0x7f86dc1630a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:15.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.914+0000 7f86e31f3640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc0d31d0 con 0x7f86dc0ce1c0 2024-04-25T10:59:15.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.914+0000 7f86e31f3640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc0cf0a0 con 0x7f86dc0cec90 2024-04-25T10:59:15.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.914+0000 7f86e31f3640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f86dc0b4a40 con 0x7f86dc0b3f20 2024-04-25T10:59:15.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.914+0000 7f86dbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86dc0ce1c0 0x7f86dc162b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:15.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.914+0000 7f86dbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86dc0ce1c0 0x7f86dc162b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41010/0 (socket says 172.21.15.69:41010) 2024-04-25T10:59:15.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.914+0000 7f86dbfff640 1 -- 172.21.15.69:0/301258837 learned_addr learned my addr 172.21.15.69:0/301258837 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:15.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.915+0000 7f86e0f8f640 1 --2- 172.21.15.69:0/301258837 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86dc0b3f20 0x7f86dc151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:15.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.915+0000 7f86dbfff640 1 -- 172.21.15.69:0/301258837 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86dc0b3f20 msgr2=0x7f86dc151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:15.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.915+0000 7f86e1790640 1 --2- 172.21.15.69:0/301258837 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86dc0cec90 0x7f86dc1630a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:15.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.915+0000 7f86dbfff640 1 --2- 172.21.15.69:0/301258837 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86dc0b3f20 0x7f86dc151a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:15.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.915+0000 7f86dbfff640 1 -- 172.21.15.69:0/301258837 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86dc0cec90 msgr2=0x7f86dc1630a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:15.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.915+0000 7f86dbfff640 1 --2- 172.21.15.69:0/301258837 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86dc0cec90 0x7f86dc1630a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:15.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.915+0000 7f86dbfff640 1 -- 172.21.15.69:0/301258837 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86dc0c7420 con 0x7f86dc0ce1c0 2024-04-25T10:59:15.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.915+0000 7f86dbfff640 1 --2- 172.21.15.69:0/301258837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86dc0ce1c0 0x7f86dc162b60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0098c0 tx=0x7f86cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:15.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.915+0000 7f86d97fa640 1 -- 172.21.15.69:0/301258837 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc042070 con 0x7f86dc0ce1c0 2024-04-25T10:59:15.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.915+0000 7f86d97fa640 1 -- 172.21.15.69:0/301258837 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86cc03e670 con 0x7f86dc0ce1c0 2024-04-25T10:59:15.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.915+0000 7f86e31f3640 1 -- 172.21.15.69:0/301258837 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86dc0c7700 con 0x7f86dc0ce1c0 2024-04-25T10:59:15.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.915+0000 7f86d97fa640 1 -- 172.21.15.69:0/301258837 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc037040 con 0x7f86dc0ce1c0 2024-04-25T10:59:15.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.915+0000 7f86e31f3640 1 -- 172.21.15.69:0/301258837 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86dc0c9130 con 0x7f86dc0ce1c0 2024-04-25T10:59:15.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.916+0000 7f86e31f3640 1 -- 172.21.15.69:0/301258837 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86dc0c7190 con 0x7f86dc0ce1c0 2024-04-25T10:59:15.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.917+0000 7f86d97fa640 1 -- 172.21.15.69:0/301258837 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f86cc045070 con 0x7f86dc0ce1c0 2024-04-25T10:59:15.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.917+0000 7f86d97fa640 1 --2- 172.21.15.69:0/301258837 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f86bc0744e0 0x7f86bc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:15.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.917+0000 7f86e0f8f640 1 --2- 172.21.15.69:0/301258837 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f86bc0744e0 0x7f86bc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:15.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.917+0000 7f86d97fa640 1 -- 172.21.15.69:0/301258837 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f86cc0b5690 con 0x7f86dc0ce1c0 2024-04-25T10:59:15.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.918+0000 7f86e0f8f640 1 --2- 172.21.15.69:0/301258837 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f86bc0744e0 0x7f86bc0769f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f86dc073a20 tx=0x7f86c4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:15.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:15.922+0000 7f86d97fa640 1 -- 172.21.15.69:0/301258837 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f86cc080200 con 0x7f86dc0ce1c0 2024-04-25T10:59:16.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.094+0000 7f86e31f3640 1 -- 172.21.15.69:0/301258837 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f86dc0d09a0 con 0x7f86dc0ce1c0 2024-04-25T10:59:16.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.095+0000 7f86d97fa640 1 -- 172.21.15.69:0/301258837 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f86dc0d09a0 con 0x7f86dc0ce1c0 2024-04-25T10:59:16.096 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-25T10:59:16.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.097+0000 7f86e31f3640 1 -- 172.21.15.69:0/301258837 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f86bc0744e0 msgr2=0x7f86bc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:16.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.097+0000 7f86e31f3640 1 --2- 172.21.15.69:0/301258837 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f86bc0744e0 0x7f86bc0769f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f86dc073a20 tx=0x7f86c4009040 comp rx=0 tx=0).stop 2024-04-25T10:59:16.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.097+0000 7f86e31f3640 1 -- 172.21.15.69:0/301258837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86dc0ce1c0 msgr2=0x7f86dc162b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:16.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.097+0000 7f86e31f3640 1 --2- 172.21.15.69:0/301258837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86dc0ce1c0 0x7f86dc162b60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0098c0 tx=0x7f86cc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:59:16.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.097+0000 7f86e31f3640 1 -- 172.21.15.69:0/301258837 shutdown_connections 2024-04-25T10:59:16.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.097+0000 7f86e31f3640 1 --2- 172.21.15.69:0/301258837 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f86bc0744e0 0x7f86bc0769f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:16.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.097+0000 7f86e31f3640 1 --2- 172.21.15.69:0/301258837 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86dc0cec90 0x7f86dc1630a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:16.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.097+0000 7f86e31f3640 1 --2- 172.21.15.69:0/301258837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86dc0ce1c0 0x7f86dc162b60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:16.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.097+0000 7f86e31f3640 1 --2- 172.21.15.69:0/301258837 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86dc0b3f20 0x7f86dc151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:16.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.097+0000 7f86e31f3640 1 -- 172.21.15.69:0/301258837 >> 172.21.15.69:0/301258837 conn(0x7f86dc09b7d0 msgr2=0x7f86dc06e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:16.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.097+0000 7f86e31f3640 1 -- 172.21.15.69:0/301258837 shutdown_connections 2024-04-25T10:59:16.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.097+0000 7f86e31f3640 1 -- 172.21.15.69:0/301258837 wait complete. 2024-04-25T10:59:16.109 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:59:16.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.215+0000 7f514675f640 1 Processor -- start 2024-04-25T10:59:16.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.215+0000 7f514675f640 1 -- start start 2024-04-25T10:59:16.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.216+0000 7f514675f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51401616a0 0x7f514015ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:16.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.216+0000 7f514675f640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f514006e030 0x7f514006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:16.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.216+0000 7f514675f640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f514006e980 con 0x7f5140163fc0 2024-04-25T10:59:16.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.216+0000 7f514675f640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f5140160980 con 0x7f514006e030 2024-04-25T10:59:16.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.216+0000 7f514675f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5140164c80 con 0x7f51401616a0 2024-04-25T10:59:16.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.216+0000 7f513ffff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51401616a0 0x7f514015ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:16.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.216+0000 7f513f7fe640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f514006e030 0x7f514006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:16.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.217+0000 7f5144cfc640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f5140163fc0 0x7f51401643d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.69:50276/0 (socket says 172.21.15.69:50276) 2024-04-25T10:59:16.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.217+0000 7f513f7fe640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f514006e030 0x7f514006e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.69:51460/0 (socket says 172.21.15.69:51460) 2024-04-25T10:59:16.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.217+0000 7f513ffff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51401616a0 0x7f514015ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.69:57932/0 (socket says 172.21.15.69:57932) 2024-04-25T10:59:16.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.217+0000 7f513f7fe640 1 -- 172.21.15.69:0/4000047458 learned_addr learned my addr 172.21.15.69:0/4000047458 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:16.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.217+0000 7f513f7fe640 1 -- 172.21.15.69:0/4000047458 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51401616a0 msgr2=0x7f514015ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:16.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.217+0000 7f513f7fe640 1 --2- 172.21.15.69:0/4000047458 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51401616a0 0x7f514015ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:16.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.217+0000 7f513f7fe640 1 -- 172.21.15.69:0/4000047458 >> v1:172.21.15.146:6789/0 conn(0x7f5140163fc0 legacy=0x7f51401643d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:16.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.217+0000 7f513f7fe640 1 -- 172.21.15.69:0/4000047458 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5140160bc0 con 0x7f514006e030 2024-04-25T10:59:16.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.217+0000 7f513ffff640 1 --2- 172.21.15.69:0/4000047458 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51401616a0 0x7f514015ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:59:16.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.218+0000 7f513f7fe640 1 --2- 172.21.15.69:0/4000047458 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f514006e030 0x7f514006e440 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5134009870 tx=0x7f513402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f998fdf5f0321d45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:16.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.218+0000 7f513effd640 1 -- 172.21.15.69:0/4000047458 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5134037070 con 0x7f514006e030 2024-04-25T10:59:16.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.218+0000 7f513effd640 1 -- 172.21.15.69:0/4000047458 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5134032440 con 0x7f514006e030 2024-04-25T10:59:16.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.218+0000 7f513effd640 1 -- 172.21.15.69:0/4000047458 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5134032810 con 0x7f514006e030 2024-04-25T10:59:16.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.218+0000 7f514675f640 1 -- 172.21.15.69:0/4000047458 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f514006e030 msgr2=0x7f514006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:16.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.218+0000 7f514675f640 1 --2- 172.21.15.69:0/4000047458 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f514006e030 0x7f514006e440 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5134009870 tx=0x7f513402a270 comp rx=0 tx=0).stop 2024-04-25T10:59:16.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.219+0000 7f514675f640 1 -- 172.21.15.69:0/4000047458 shutdown_connections 2024-04-25T10:59:16.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.219+0000 7f514675f640 1 --2- 172.21.15.69:0/4000047458 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f514006e030 0x7f514006e440 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:16.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.219+0000 7f514675f640 1 --2- 172.21.15.69:0/4000047458 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51401616a0 0x7f514015ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:16.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.219+0000 7f514675f640 1 -- 172.21.15.69:0/4000047458 >> 172.21.15.69:0/4000047458 conn(0x7f51400d27c0 msgr2=0x7f51400d2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:16.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.219+0000 7f514675f640 1 -- 172.21.15.69:0/4000047458 shutdown_connections 2024-04-25T10:59:16.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.219+0000 7f514675f640 1 -- 172.21.15.69:0/4000047458 wait complete. 2024-04-25T10:59:16.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.220+0000 7f514675f640 1 Processor -- start 2024-04-25T10:59:16.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.220+0000 7f514675f640 1 -- start start 2024-04-25T10:59:16.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.222+0000 7f514675f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f514006e030 0x7f51401faa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:16.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.222+0000 7f514675f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51401616a0 0x7f51400ba800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:16.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.222+0000 7f514675f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5140163fc0 0x7f514014fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:16.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.222+0000 7f514675f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f51400b77c0 con 0x7f5140163fc0 2024-04-25T10:59:16.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.222+0000 7f514675f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5140161180 con 0x7f514006e030 2024-04-25T10:59:16.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.222+0000 7f514675f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f51400b7940 con 0x7f51401616a0 2024-04-25T10:59:16.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.222+0000 7f5144cfc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5140163fc0 0x7f514014fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:16.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.222+0000 7f5144cfc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5140163fc0 0x7f514014fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41024/0 (socket says 172.21.15.69:41024) 2024-04-25T10:59:16.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.222+0000 7f5144cfc640 1 -- 172.21.15.69:0/2117107712 learned_addr learned my addr 172.21.15.69:0/2117107712 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:16.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.222+0000 7f513f7fe640 1 --2- 172.21.15.69:0/2117107712 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51401616a0 0x7f51400ba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:16.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.222+0000 7f513ffff640 1 --2- 172.21.15.69:0/2117107712 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f514006e030 0x7f51401faa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:16.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.222+0000 7f5144cfc640 1 -- 172.21.15.69:0/2117107712 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51401616a0 msgr2=0x7f51400ba800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:16.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.222+0000 7f5144cfc640 1 --2- 172.21.15.69:0/2117107712 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51401616a0 0x7f51400ba800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:16.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.222+0000 7f5144cfc640 1 -- 172.21.15.69:0/2117107712 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f514006e030 msgr2=0x7f51401faa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:16.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.222+0000 7f5144cfc640 1 --2- 172.21.15.69:0/2117107712 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f514006e030 0x7f51401faa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:16.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.222+0000 7f5144cfc640 1 -- 172.21.15.69:0/2117107712 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51401faf80 con 0x7f5140163fc0 2024-04-25T10:59:16.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.222+0000 7f5144cfc640 1 --2- 172.21.15.69:0/2117107712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5140163fc0 0x7f514014fc50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f513800c6c0 tx=0x7f513800cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:16.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.222+0000 7f513cff9640 1 -- 172.21.15.69:0/2117107712 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5138018070 con 0x7f5140163fc0 2024-04-25T10:59:16.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.223+0000 7f513cff9640 1 -- 172.21.15.69:0/2117107712 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5138004d60 con 0x7f5140163fc0 2024-04-25T10:59:16.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.223+0000 7f514675f640 1 -- 172.21.15.69:0/2117107712 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51400cefc0 con 0x7f5140163fc0 2024-04-25T10:59:16.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.223+0000 7f513cff9640 1 -- 172.21.15.69:0/2117107712 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513800a840 con 0x7f5140163fc0 2024-04-25T10:59:16.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.223+0000 7f514675f640 1 -- 172.21.15.69:0/2117107712 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51401ea510 con 0x7f5140163fc0 2024-04-25T10:59:16.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.224+0000 7f514675f640 1 -- 172.21.15.69:0/2117107712 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51401e9fa0 con 0x7f5140163fc0 2024-04-25T10:59:16.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.224+0000 7f513cff9640 1 -- 172.21.15.69:0/2117107712 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f513801b070 con 0x7f5140163fc0 2024-04-25T10:59:16.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.225+0000 7f513cff9640 1 --2- 172.21.15.69:0/2117107712 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f512c0744b0 0x7f512c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:16.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.225+0000 7f513cff9640 1 -- 172.21.15.69:0/2117107712 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f5138094410 con 0x7f5140163fc0 2024-04-25T10:59:16.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.225+0000 7f513ffff640 1 --2- 172.21.15.69:0/2117107712 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f512c0744b0 0x7f512c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:16.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.225+0000 7f513ffff640 1 --2- 172.21.15.69:0/2117107712 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f512c0744b0 0x7f512c0769c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5130009560 tx=0x7f5130009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:16.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.229+0000 7f513cff9640 1 -- 172.21.15.69:0/2117107712 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f51401faf80 con 0x7f5140163fc0 2024-04-25T10:59:16.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.408+0000 7f514675f640 1 -- 172.21.15.69:0/2117107712 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f51400d3980 con 0x7f5140163fc0 2024-04-25T10:59:16.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.408+0000 7f513cff9640 1 -- 172.21.15.69:0/2117107712 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f514006e980 con 0x7f5140163fc0 2024-04-25T10:59:16.409 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:16.409 INFO:teuthology.orchestra.run.smithi069.stdout:[ 2024-04-25T10:59:16.409 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T10:59:16.409 INFO:teuthology.orchestra.run.smithi069.stdout: "poolnum": 1, 2024-04-25T10:59:16.410 INFO:teuthology.orchestra.run.smithi069.stdout: "poolname": ".mgr" 2024-04-25T10:59:16.410 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T10:59:16.410 INFO:teuthology.orchestra.run.smithi069.stdout:] 2024-04-25T10:59:16.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.410+0000 7f514675f640 1 -- 172.21.15.69:0/2117107712 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f512c0744b0 msgr2=0x7f512c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:16.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.410+0000 7f514675f640 1 --2- 172.21.15.69:0/2117107712 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f512c0744b0 0x7f512c0769c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5130009560 tx=0x7f5130009190 comp rx=0 tx=0).stop 2024-04-25T10:59:16.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.410+0000 7f514675f640 1 -- 172.21.15.69:0/2117107712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5140163fc0 msgr2=0x7f514014fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:16.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.410+0000 7f514675f640 1 --2- 172.21.15.69:0/2117107712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5140163fc0 0x7f514014fc50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f513800c6c0 tx=0x7f513800cb80 comp rx=0 tx=0).stop 2024-04-25T10:59:16.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.410+0000 7f514675f640 1 -- 172.21.15.69:0/2117107712 shutdown_connections 2024-04-25T10:59:16.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.410+0000 7f514675f640 1 --2- 172.21.15.69:0/2117107712 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f512c0744b0 0x7f512c0769c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:16.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.410+0000 7f514675f640 1 --2- 172.21.15.69:0/2117107712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5140163fc0 0x7f514014fc50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:16.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.410+0000 7f514675f640 1 --2- 172.21.15.69:0/2117107712 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51401616a0 0x7f51400ba800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:16.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.410+0000 7f514675f640 1 --2- 172.21.15.69:0/2117107712 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f514006e030 0x7f51401faa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:16.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.410+0000 7f514675f640 1 -- 172.21.15.69:0/2117107712 >> 172.21.15.69:0/2117107712 conn(0x7f51400d27c0 msgr2=0x7f5140160100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:16.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.411+0000 7f514675f640 1 -- 172.21.15.69:0/2117107712 shutdown_connections 2024-04-25T10:59:16.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.411+0000 7f514675f640 1 -- 172.21.15.69:0/2117107712 wait complete. 2024-04-25T10:59:16.422 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:59:16.422 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:59:16.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.532+0000 7f6a76820640 1 Processor -- start 2024-04-25T10:59:16.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.532+0000 7f6a76820640 1 -- start start 2024-04-25T10:59:16.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.533+0000 7f6a76820640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6a700b8290 0x7f6a70160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:16.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.533+0000 7f6a76820640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a701615f0 con 0x7f6a70164b00 2024-04-25T10:59:16.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.533+0000 7f6a76820640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a70164020 con 0x7f6a7015df80 2024-04-25T10:59:16.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.533+0000 7f6a76820640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f6a70164670 con 0x7f6a700b8290 2024-04-25T10:59:16.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.534+0000 7f6a6ffff640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f6a70164b00 0x7f6a700b7b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:57076/0 (socket says 172.21.15.69:57076) 2024-04-25T10:59:16.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.534+0000 7f6a6ffff640 1 -- 172.21.15.69:0/3385936523 learned_addr learned my addr 172.21.15.69:0/3385936523 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:16.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.534+0000 7f6a6f7fe640 1 --2- 172.21.15.69:0/3385936523 >> v2:172.21.15.146:3300/0 conn(0x7f6a700b8290 0x7f6a70160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:16.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.534+0000 7f6a6effd640 1 -- 172.21.15.69:0/3385936523 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2478583562 0 0) 0x7f6a701615f0 con 0x7f6a70164b00 2024-04-25T10:59:16.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.534+0000 7f6a6effd640 1 -- 172.21.15.69:0/3385936523 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a58003520 con 0x7f6a70164b00 2024-04-25T10:59:16.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.534+0000 7f6a6effd640 1 -- 172.21.15.69:0/3385936523 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2059013176 0 0) 0x7f6a58003520 con 0x7f6a70164b00 2024-04-25T10:59:16.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.534+0000 7f6a6effd640 1 -- 172.21.15.69:0/3385936523 >> v2:172.21.15.146:3300/0 conn(0x7f6a700b8290 msgr2=0x7f6a70160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:16.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.534+0000 7f6a6effd640 1 --2- 172.21.15.69:0/3385936523 >> v2:172.21.15.146:3300/0 conn(0x7f6a700b8290 0x7f6a70160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:16.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.534+0000 7f6a6effd640 1 -- 172.21.15.69:0/3385936523 >> v1:172.21.15.146:6789/0 conn(0x7f6a7015df80 legacy=0x7f6a70160dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:16.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.534+0000 7f6a6effd640 1 -- 172.21.15.69:0/3385936523 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a70164300 con 0x7f6a70164b00 2024-04-25T10:59:16.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.534+0000 7f6a6effd640 1 -- 172.21.15.69:0/3385936523 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7f6a60001670 con 0x7f6a70164b00 2024-04-25T10:59:16.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.534+0000 7f6a6effd640 1 -- 172.21.15.69:0/3385936523 >> v1:172.21.15.69:6789/0 conn(0x7f6a70164b00 legacy=0x7f6a700b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:16.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.534+0000 7f6a6effd640 1 --2- 172.21.15.69:0/3385936523 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a580039b0 0x7f6a58003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:16.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.534+0000 7f6a6effd640 1 --2- 172.21.15.69:0/3385936523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a58004800 0x7f6a58004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:16.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.534+0000 7f6a6effd640 1 --2- 172.21.15.69:0/3385936523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a58005270 0x7f6a5800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:16.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.534+0000 7f6a6effd640 1 -- 172.21.15.69:0/3385936523 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6a70164670 con 0x7f6a58005270 2024-04-25T10:59:16.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.534+0000 7f6a6effd640 1 -- 172.21.15.69:0/3385936523 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5800c190 con 0x7f6a58004800 2024-04-25T10:59:16.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.534+0000 7f6a6effd640 1 -- 172.21.15.69:0/3385936523 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6a5800c390 con 0x7f6a580039b0 2024-04-25T10:59:16.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.534+0000 7f6a74dbd640 1 --2- 172.21.15.69:0/3385936523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a58005270 0x7f6a5800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:16.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.534+0000 7f6a6ffff640 1 --2- 172.21.15.69:0/3385936523 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a580039b0 0x7f6a58003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:16.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.534+0000 7f6a6f7fe640 1 --2- 172.21.15.69:0/3385936523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a58004800 0x7f6a58004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:16.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.535+0000 7f6a74dbd640 1 -- 172.21.15.69:0/3385936523 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a580039b0 msgr2=0x7f6a58003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:16.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.535+0000 7f6a74dbd640 1 --2- 172.21.15.69:0/3385936523 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a580039b0 0x7f6a58003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:16.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.535+0000 7f6a74dbd640 1 -- 172.21.15.69:0/3385936523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a58004800 msgr2=0x7f6a58004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:16.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.535+0000 7f6a74dbd640 1 --2- 172.21.15.69:0/3385936523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a58004800 0x7f6a58004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:16.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.535+0000 7f6a74dbd640 1 -- 172.21.15.69:0/3385936523 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a58003520 con 0x7f6a58005270 2024-04-25T10:59:16.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.535+0000 7f6a74dbd640 1 -- 172.21.15.69:0/3385936523 reap_dead start 2024-04-25T10:59:16.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.535+0000 7f6a74dbd640 1 --2- 172.21.15.69:0/3385936523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a58005270 0x7f6a5800b9c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6a68001e20 tx=0x7f6a58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519e7d3531e3aaa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:16.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.535+0000 7f6a6effd640 1 -- 172.21.15.69:0/3385936523 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a68001a70 con 0x7f6a58005270 2024-04-25T10:59:16.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.535+0000 7f6a6effd640 1 -- 172.21.15.69:0/3385936523 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a6802ae40 con 0x7f6a58005270 2024-04-25T10:59:16.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.535+0000 7f6a6effd640 1 -- 172.21.15.69:0/3385936523 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a68003bb0 con 0x7f6a58005270 2024-04-25T10:59:16.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.536+0000 7f6a76820640 1 -- 172.21.15.69:0/3385936523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a58005270 msgr2=0x7f6a5800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:16.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.536+0000 7f6a76820640 1 --2- 172.21.15.69:0/3385936523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a58005270 0x7f6a5800b9c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6a68001e20 tx=0x7f6a58005830 comp rx=0 tx=0).stop 2024-04-25T10:59:16.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.536+0000 7f6a76820640 1 -- 172.21.15.69:0/3385936523 shutdown_connections 2024-04-25T10:59:16.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.536+0000 7f6a76820640 1 --2- 172.21.15.69:0/3385936523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a58005270 0x7f6a5800b9c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:16.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.536+0000 7f6a76820640 1 -- 172.21.15.69:0/3385936523 >> 172.21.15.69:0/3385936523 conn(0x7f6a700d2bc0 msgr2=0x7f6a700d2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:16.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.536+0000 7f6a76820640 1 -- 172.21.15.69:0/3385936523 shutdown_connections 2024-04-25T10:59:16.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.536+0000 7f6a76820640 1 -- 172.21.15.69:0/3385936523 wait complete. 2024-04-25T10:59:16.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.538+0000 7f6a76820640 1 Processor -- start 2024-04-25T10:59:16.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.538+0000 7f6a76820640 1 -- start start 2024-04-25T10:59:16.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.539+0000 7f6a76820640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a58005270 0x7f6a70153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:16.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.539+0000 7f6a76820640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a70159040 0x7f6a70148200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:16.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.539+0000 7f6a76820640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a701f62c0 0x7f6a70157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:16.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.539+0000 7f6a76820640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6a700b7f10 con 0x7f6a70159040 2024-04-25T10:59:16.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.539+0000 7f6a76820640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6a700b7d90 con 0x7f6a58005270 2024-04-25T10:59:16.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.539+0000 7f6a76820640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6a700b8090 con 0x7f6a701f62c0 2024-04-25T10:59:16.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.539+0000 7f6a6f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a70159040 0x7f6a70148200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:16.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.539+0000 7f6a6f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a70159040 0x7f6a70148200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41042/0 (socket says 172.21.15.69:41042) 2024-04-25T10:59:16.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.539+0000 7f6a6f7fe640 1 -- 172.21.15.69:0/127272136 learned_addr learned my addr 172.21.15.69:0/127272136 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:16.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.540+0000 7f6a6ffff640 1 --2- 172.21.15.69:0/127272136 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a58005270 0x7f6a70153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:16.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.540+0000 7f6a74dbd640 1 --2- 172.21.15.69:0/127272136 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a701f62c0 0x7f6a70157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:16.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.540+0000 7f6a6f7fe640 1 -- 172.21.15.69:0/127272136 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a701f62c0 msgr2=0x7f6a70157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:16.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.540+0000 7f6a6f7fe640 1 --2- 172.21.15.69:0/127272136 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a701f62c0 0x7f6a70157110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:16.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.540+0000 7f6a6f7fe640 1 -- 172.21.15.69:0/127272136 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a58005270 msgr2=0x7f6a70153460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:16.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.540+0000 7f6a6f7fe640 1 --2- 172.21.15.69:0/127272136 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a58005270 0x7f6a70153460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:16.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.540+0000 7f6a6f7fe640 1 -- 172.21.15.69:0/127272136 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a70156070 con 0x7f6a70159040 2024-04-25T10:59:16.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.540+0000 7f6a6f7fe640 1 --2- 172.21.15.69:0/127272136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a70159040 0x7f6a70148200 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6a640118e0 tx=0x7f6a64011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:16.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.540+0000 7f6a6cff9640 1 -- 172.21.15.69:0/127272136 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6400faf0 con 0x7f6a70159040 2024-04-25T10:59:16.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.540+0000 7f6a6cff9640 1 -- 172.21.15.69:0/127272136 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a6400fc90 con 0x7f6a70159040 2024-04-25T10:59:16.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.540+0000 7f6a76820640 1 -- 172.21.15.69:0/127272136 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a70156350 con 0x7f6a70159040 2024-04-25T10:59:16.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.540+0000 7f6a6cff9640 1 -- 172.21.15.69:0/127272136 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a64014440 con 0x7f6a70159040 2024-04-25T10:59:16.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.540+0000 7f6a76820640 1 -- 172.21.15.69:0/127272136 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a70156910 con 0x7f6a70159040 2024-04-25T10:59:16.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.541+0000 7f6a76820640 1 -- 172.21.15.69:0/127272136 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a70050740 con 0x7f6a70159040 2024-04-25T10:59:16.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.542+0000 7f6a6cff9640 1 -- 172.21.15.69:0/127272136 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6a64012040 con 0x7f6a70159040 2024-04-25T10:59:16.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.542+0000 7f6a6cff9640 1 --2- 172.21.15.69:0/127272136 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f6a5c0744e0 0x7f6a5c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:16.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.542+0000 7f6a6ffff640 1 --2- 172.21.15.69:0/127272136 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f6a5c0744e0 0x7f6a5c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:16.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.542+0000 7f6a6cff9640 1 -- 172.21.15.69:0/127272136 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f6a64095640 con 0x7f6a70159040 2024-04-25T10:59:16.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.543+0000 7f6a6ffff640 1 --2- 172.21.15.69:0/127272136 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f6a5c0744e0 0x7f6a5c0769f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6a60001050 tx=0x7f6a60009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:16.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.547+0000 7f6a6cff9640 1 -- 172.21.15.69:0/127272136 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6a64019030 con 0x7f6a70159040 2024-04-25T10:59:16.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:16.723+0000 7f6a76820640 1 -- 172.21.15.69:0/127272136 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f6a700d4170 con 0x7f6a70159040 2024-04-25T10:59:17.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.609+0000 7f6a6cff9640 1 -- 172.21.15.69:0/127272136 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f6a64060120 con 0x7f6a70159040 2024-04-25T10:59:17.609 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_metadata' created 2024-04-25T10:59:17.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.611+0000 7f6a76820640 1 -- 172.21.15.69:0/127272136 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f6a5c0744e0 msgr2=0x7f6a5c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:17.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.611+0000 7f6a76820640 1 --2- 172.21.15.69:0/127272136 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f6a5c0744e0 0x7f6a5c0769f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6a60001050 tx=0x7f6a60009040 comp rx=0 tx=0).stop 2024-04-25T10:59:17.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.611+0000 7f6a76820640 1 -- 172.21.15.69:0/127272136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a70159040 msgr2=0x7f6a70148200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:17.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.611+0000 7f6a76820640 1 --2- 172.21.15.69:0/127272136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a70159040 0x7f6a70148200 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6a640118e0 tx=0x7f6a64011da0 comp rx=0 tx=0).stop 2024-04-25T10:59:17.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.611+0000 7f6a76820640 1 -- 172.21.15.69:0/127272136 shutdown_connections 2024-04-25T10:59:17.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.611+0000 7f6a76820640 1 --2- 172.21.15.69:0/127272136 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f6a5c0744e0 0x7f6a5c0769f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:17.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.611+0000 7f6a76820640 1 --2- 172.21.15.69:0/127272136 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a701f62c0 0x7f6a70157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:17.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.611+0000 7f6a76820640 1 --2- 172.21.15.69:0/127272136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a70159040 0x7f6a70148200 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:17.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.611+0000 7f6a76820640 1 --2- 172.21.15.69:0/127272136 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a58005270 0x7f6a70153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:17.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.611+0000 7f6a76820640 1 -- 172.21.15.69:0/127272136 >> 172.21.15.69:0/127272136 conn(0x7f6a700d2bc0 msgr2=0x7f6a7015e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:17.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.611+0000 7f6a76820640 1 -- 172.21.15.69:0/127272136 shutdown_connections 2024-04-25T10:59:17.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.611+0000 7f6a76820640 1 -- 172.21.15.69:0/127272136 wait complete. 2024-04-25T10:59:17.628 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:59:17.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.744+0000 7fbba51eb640 1 Processor -- start 2024-04-25T10:59:17.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.744+0000 7fbba51eb640 1 -- start start 2024-04-25T10:59:17.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.746+0000 7fbba51eb640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbba00d0e00 0x7fbba00d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:17.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.746+0000 7fbba51eb640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbba006c5c0 0x7fbba006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:17.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.746+0000 7fbba51eb640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbba006cf10 con 0x7fbba00b7810 2024-04-25T10:59:17.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.746+0000 7fbba51eb640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fbba00d5a60 con 0x7fbba006c5c0 2024-04-25T10:59:17.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.746+0000 7fbba51eb640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbba00d1850 con 0x7fbba00d0e00 2024-04-25T10:59:17.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.746+0000 7fbb9e59c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbba006c5c0 0x7fbba006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:17.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.746+0000 7fbb9ed9d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbba00d0e00 0x7fbba00d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:17.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.746+0000 7fbb9f59e640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fbba00b7810 0x7fbba00b7c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.69:50296/0 (socket says 172.21.15.69:50296) 2024-04-25T10:59:17.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.746+0000 7fbb9e59c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbba006c5c0 0x7fbba006c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.69:51516/0 (socket says 172.21.15.69:51516) 2024-04-25T10:59:17.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.746+0000 7fbb9f59e640 1 -- 172.21.15.69:0/1614349638 learned_addr learned my addr 172.21.15.69:0/1614349638 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:17.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.746+0000 7fbb9ed9d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbba00d0e00 0x7fbba00d1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.69:57950/0 (socket says 172.21.15.69:57950) 2024-04-25T10:59:17.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.746+0000 7fbb9e59c640 1 -- 172.21.15.69:0/1614349638 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbba00d0e00 msgr2=0x7fbba00d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:17.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.746+0000 7fbb9e59c640 1 --2- 172.21.15.69:0/1614349638 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbba00d0e00 0x7fbba00d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:17.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.746+0000 7fbb9e59c640 1 -- 172.21.15.69:0/1614349638 >> v1:172.21.15.146:6789/0 conn(0x7fbba00b7810 legacy=0x7fbba00b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:17.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.746+0000 7fbb9e59c640 1 -- 172.21.15.69:0/1614349638 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba0160de0 con 0x7fbba006c5c0 2024-04-25T10:59:17.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.746+0000 7fbb9ed9d640 1 --2- 172.21.15.69:0/1614349638 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbba00d0e00 0x7fbba00d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:59:17.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.747+0000 7fbb9e59c640 1 --2- 172.21.15.69:0/1614349638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbba006c5c0 0x7fbba006c9d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbb940097a0 tx=0x7fbb9402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f3ad5eb827bb709 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:17.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.747+0000 7fbb9dd9b640 1 -- 172.21.15.69:0/1614349638 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb94037070 con 0x7fbba006c5c0 2024-04-25T10:59:17.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.747+0000 7fbb9dd9b640 1 -- 172.21.15.69:0/1614349638 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb94032440 con 0x7fbba006c5c0 2024-04-25T10:59:17.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.747+0000 7fbb9dd9b640 1 -- 172.21.15.69:0/1614349638 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb94032810 con 0x7fbba006c5c0 2024-04-25T10:59:17.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.748+0000 7fbba51eb640 1 -- 172.21.15.69:0/1614349638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbba006c5c0 msgr2=0x7fbba006c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:17.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.748+0000 7fbba51eb640 1 --2- 172.21.15.69:0/1614349638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbba006c5c0 0x7fbba006c9d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbb940097a0 tx=0x7fbb9402a180 comp rx=0 tx=0).stop 2024-04-25T10:59:17.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.748+0000 7fbba51eb640 1 -- 172.21.15.69:0/1614349638 shutdown_connections 2024-04-25T10:59:17.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.748+0000 7fbba51eb640 1 --2- 172.21.15.69:0/1614349638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbba006c5c0 0x7fbba006c9d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:17.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.748+0000 7fbba51eb640 1 --2- 172.21.15.69:0/1614349638 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbba00d0e00 0x7fbba00d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:17.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.748+0000 7fbba51eb640 1 -- 172.21.15.69:0/1614349638 >> 172.21.15.69:0/1614349638 conn(0x7fbba00d5d90 msgr2=0x7fbba00d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:17.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.748+0000 7fbba51eb640 1 -- 172.21.15.69:0/1614349638 shutdown_connections 2024-04-25T10:59:17.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.748+0000 7fbba51eb640 1 -- 172.21.15.69:0/1614349638 wait complete. 2024-04-25T10:59:17.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.750+0000 7fbba51eb640 1 Processor -- start 2024-04-25T10:59:17.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.750+0000 7fbba51eb640 1 -- start start 2024-04-25T10:59:17.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.751+0000 7fbba51eb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbba006c5c0 0x7fbba00cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:17.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.751+0000 7fbba51eb640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbba00b7810 0x7fbba01fb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:17.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.751+0000 7fbba51eb640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbba00d0e00 0x7fbba00b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:17.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.751+0000 7fbba51eb640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbba0160ad0 con 0x7fbba006c5c0 2024-04-25T10:59:17.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.751+0000 7fbba51eb640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbba01613a0 con 0x7fbba00d0e00 2024-04-25T10:59:17.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.751+0000 7fbba51eb640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbba00b5320 con 0x7fbba00b7810 2024-04-25T10:59:17.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.752+0000 7fbb9ed9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbba006c5c0 0x7fbba00cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:17.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.752+0000 7fbb9ed9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbba006c5c0 0x7fbba00cbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41046/0 (socket says 172.21.15.69:41046) 2024-04-25T10:59:17.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.752+0000 7fbb9ed9d640 1 -- 172.21.15.69:0/3818120157 learned_addr learned my addr 172.21.15.69:0/3818120157 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:17.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.752+0000 7fbb9e59c640 1 --2- 172.21.15.69:0/3818120157 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbba00b7810 0x7fbba01fb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:17.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.752+0000 7fbb9f59e640 1 --2- 172.21.15.69:0/3818120157 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbba00d0e00 0x7fbba00b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:17.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.752+0000 7fbb9ed9d640 1 -- 172.21.15.69:0/3818120157 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbba00b7810 msgr2=0x7fbba01fb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:17.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.752+0000 7fbb9ed9d640 1 --2- 172.21.15.69:0/3818120157 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbba00b7810 0x7fbba01fb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:17.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.752+0000 7fbb9ed9d640 1 -- 172.21.15.69:0/3818120157 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbba00d0e00 msgr2=0x7fbba00b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:17.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.752+0000 7fbb9ed9d640 1 --2- 172.21.15.69:0/3818120157 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbba00d0e00 0x7fbba00b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:17.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.752+0000 7fbb9ed9d640 1 -- 172.21.15.69:0/3818120157 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba0155ec0 con 0x7fbba006c5c0 2024-04-25T10:59:17.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.752+0000 7fbb9e59c640 1 --2- 172.21.15.69:0/3818120157 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbba00b7810 0x7fbba01fb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:17.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.752+0000 7fbb9f59e640 1 --2- 172.21.15.69:0/3818120157 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbba00d0e00 0x7fbba00b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:17.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.752+0000 7fbb9ed9d640 1 --2- 172.21.15.69:0/3818120157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbba006c5c0 0x7fbba00cbde0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbb90009ec0 tx=0x7fbb9000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:17.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.752+0000 7fbb877fe640 1 -- 172.21.15.69:0/3818120157 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb90018020 con 0x7fbba006c5c0 2024-04-25T10:59:17.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.752+0000 7fbb877fe640 1 -- 172.21.15.69:0/3818120157 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb90014680 con 0x7fbba006c5c0 2024-04-25T10:59:17.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.752+0000 7fbba51eb640 1 -- 172.21.15.69:0/3818120157 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbba01fb950 con 0x7fbba006c5c0 2024-04-25T10:59:17.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.752+0000 7fbb877fe640 1 -- 172.21.15.69:0/3818120157 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb90014a50 con 0x7fbba006c5c0 2024-04-25T10:59:17.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.752+0000 7fbba51eb640 1 -- 172.21.15.69:0/3818120157 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbba00cad40 con 0x7fbba006c5c0 2024-04-25T10:59:17.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.753+0000 7fbba51eb640 1 -- 172.21.15.69:0/3818120157 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbba00b8fc0 con 0x7fbba006c5c0 2024-04-25T10:59:17.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.754+0000 7fbb877fe640 1 -- 172.21.15.69:0/3818120157 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbb90005d10 con 0x7fbba006c5c0 2024-04-25T10:59:17.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.755+0000 7fbb877fe640 1 --2- 172.21.15.69:0/3818120157 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fbb8c0744e0 0x7fbb8c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:17.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.755+0000 7fbb9e59c640 1 --2- 172.21.15.69:0/3818120157 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fbb8c0744e0 0x7fbb8c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:17.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.755+0000 7fbb877fe640 1 -- 172.21.15.69:0/3818120157 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7fbb900962e0 con 0x7fbba006c5c0 2024-04-25T10:59:17.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.755+0000 7fbb9e59c640 1 --2- 172.21.15.69:0/3818120157 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fbb8c0744e0 0x7fbb8c0769f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbb94034000 tx=0x7fbb9402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:17.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.759+0000 7fbb877fe640 1 -- 172.21.15.69:0/3818120157 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbba00b8fc0 con 0x7fbba006c5c0 2024-04-25T10:59:17.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:17.935+0000 7fbba51eb640 1 -- 172.21.15.69:0/3818120157 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fbba0050740 con 0x7fbba006c5c0 2024-04-25T10:59:18.074 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:59:18.077 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:59:18.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.614+0000 7fbb877fe640 1 -- 172.21.15.69:0/3818120157 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7fbb90022030 con 0x7fbba006c5c0 2024-04-25T10:59:18.615 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_data' created 2024-04-25T10:59:18.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.617+0000 7fbba51eb640 1 -- 172.21.15.69:0/3818120157 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fbb8c0744e0 msgr2=0x7fbb8c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:18.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.617+0000 7fbba51eb640 1 --2- 172.21.15.69:0/3818120157 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fbb8c0744e0 0x7fbb8c0769f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbb94034000 tx=0x7fbb9402e000 comp rx=0 tx=0).stop 2024-04-25T10:59:18.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.617+0000 7fbba51eb640 1 -- 172.21.15.69:0/3818120157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbba006c5c0 msgr2=0x7fbba00cbde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:18.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.617+0000 7fbba51eb640 1 --2- 172.21.15.69:0/3818120157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbba006c5c0 0x7fbba00cbde0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbb90009ec0 tx=0x7fbb9000c490 comp rx=0 tx=0).stop 2024-04-25T10:59:18.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.617+0000 7fbba51eb640 1 -- 172.21.15.69:0/3818120157 shutdown_connections 2024-04-25T10:59:18.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.617+0000 7fbba51eb640 1 --2- 172.21.15.69:0/3818120157 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fbb8c0744e0 0x7fbb8c0769f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:18.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.617+0000 7fbba51eb640 1 --2- 172.21.15.69:0/3818120157 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbba00d0e00 0x7fbba00b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:18.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.617+0000 7fbba51eb640 1 --2- 172.21.15.69:0/3818120157 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbba00b7810 0x7fbba01fb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:18.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.617+0000 7fbba51eb640 1 --2- 172.21.15.69:0/3818120157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbba006c5c0 0x7fbba00cbde0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:18.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.617+0000 7fbba51eb640 1 -- 172.21.15.69:0/3818120157 >> 172.21.15.69:0/3818120157 conn(0x7fbba00d5d90 msgr2=0x7fbba01641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:18.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.617+0000 7fbba51eb640 1 -- 172.21.15.69:0/3818120157 shutdown_connections 2024-04-25T10:59:18.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.617+0000 7fbba51eb640 1 -- 172.21.15.69:0/3818120157 wait complete. 2024-04-25T10:59:18.638 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:59:18.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.748+0000 7f2204b01640 1 Processor -- start 2024-04-25T10:59:18.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.748+0000 7f2204b01640 1 -- start start 2024-04-25T10:59:18.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.750+0000 7f2204b01640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f22000cd9a0 0x7f220015f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:18.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.750+0000 7f2204b01640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22000b0af0 0x7f220006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:18.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.750+0000 7f2204b01640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f220006df20 con 0x7f220006db10 2024-04-25T10:59:18.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.750+0000 7f2204b01640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f22000ceab0 con 0x7f22000cd9a0 2024-04-25T10:59:18.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.750+0000 7f2204b01640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f22000cdea0 con 0x7f22000b0af0 2024-04-25T10:59:18.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.750+0000 7f21fed9d640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f220006db10 0x7f2200161ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:57092/0 (socket says 172.21.15.69:57092) 2024-04-25T10:59:18.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.750+0000 7f21fed9d640 1 -- 172.21.15.69:0/3263549749 learned_addr learned my addr 172.21.15.69:0/3263549749 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:18.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.750+0000 7f21fe59c640 1 --2- 172.21.15.69:0/3263549749 >> v2:172.21.15.146:3300/0 conn(0x7f22000cd9a0 0x7f220015f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:18.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.750+0000 7f21fdd9b640 1 --2- 172.21.15.69:0/3263549749 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22000b0af0 0x7f220006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:18.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.750+0000 7f21fd59a640 1 -- 172.21.15.69:0/3263549749 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2164353439 0 0) 0x7f21f8001110 con 0x7f220006db10 2024-04-25T10:59:18.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.750+0000 7f21fd59a640 1 -- 172.21.15.69:0/3263549749 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f21e8003520 con 0x7f220006db10 2024-04-25T10:59:18.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.751+0000 7f21fd59a640 1 -- 172.21.15.69:0/3263549749 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3813545195 0 0) 0x7f21e8003520 con 0x7f220006db10 2024-04-25T10:59:18.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.751+0000 7f21fd59a640 1 -- 172.21.15.69:0/3263549749 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22000b0af0 msgr2=0x7f220006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:18.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.751+0000 7f21fd59a640 1 --2- 172.21.15.69:0/3263549749 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22000b0af0 0x7f220006d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:18.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.751+0000 7f21fd59a640 1 -- 172.21.15.69:0/3263549749 >> v2:172.21.15.146:3300/0 conn(0x7f22000cd9a0 msgr2=0x7f220015f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:18.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.751+0000 7f21fd59a640 1 --2- 172.21.15.69:0/3263549749 >> v2:172.21.15.146:3300/0 conn(0x7f22000cd9a0 0x7f220015f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:18.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.751+0000 7f21fd59a640 1 -- 172.21.15.69:0/3263549749 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22000ce180 con 0x7f220006db10 2024-04-25T10:59:18.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.751+0000 7f21fe59c640 1 --2- 172.21.15.69:0/3263549749 >> v2:172.21.15.146:3300/0 conn(0x7f22000cd9a0 0x7f220015f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:59:18.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.751+0000 7f21fd59a640 1 -- 172.21.15.69:0/3263549749 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7f21f8001840 con 0x7f220006db10 2024-04-25T10:59:18.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.751+0000 7f21fd59a640 1 -- 172.21.15.69:0/3263549749 >> v1:172.21.15.69:6789/0 conn(0x7f220006db10 legacy=0x7f2200161ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:18.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.751+0000 7f21fd59a640 1 --2- 172.21.15.69:0/3263549749 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22000b0af0 0x7f21e80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:18.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.751+0000 7f21fd59a640 1 --2- 172.21.15.69:0/3263549749 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21e8003f30 0x7f21e8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:18.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.751+0000 7f21fd59a640 1 --2- 172.21.15.69:0/3263549749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21e8004d40 0x7f21e8005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:18.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.751+0000 7f21fd59a640 1 -- 172.21.15.69:0/3263549749 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f22000ceab0 con 0x7f21e8004d40 2024-04-25T10:59:18.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.751+0000 7f21fd59a640 1 -- 172.21.15.69:0/3263549749 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f22000cdea0 con 0x7f21e8003f30 2024-04-25T10:59:18.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.751+0000 7f21fd59a640 1 -- 172.21.15.69:0/3263549749 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f21e800ba10 con 0x7f22000b0af0 2024-04-25T10:59:18.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.751+0000 7f21fed9d640 1 --2- 172.21.15.69:0/3263549749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21e8004d40 0x7f21e8005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:18.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.751+0000 7f21fe59c640 1 --2- 172.21.15.69:0/3263549749 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22000b0af0 0x7f21e80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:18.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.751+0000 7f21fdd9b640 1 --2- 172.21.15.69:0/3263549749 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21e8003f30 0x7f21e8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:18.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.751+0000 7f21fed9d640 1 -- 172.21.15.69:0/3263549749 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22000b0af0 msgr2=0x7f21e80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:18.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.751+0000 7f21fed9d640 1 --2- 172.21.15.69:0/3263549749 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22000b0af0 0x7f21e80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:18.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.751+0000 7f21fed9d640 1 -- 172.21.15.69:0/3263549749 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21e8003f30 msgr2=0x7f21e8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:18.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.751+0000 7f21fed9d640 1 --2- 172.21.15.69:0/3263549749 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21e8003f30 0x7f21e8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:18.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.751+0000 7f21fed9d640 1 -- 172.21.15.69:0/3263549749 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e8003520 con 0x7f21e8004d40 2024-04-25T10:59:18.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.752+0000 7f21fed9d640 1 --2- 172.21.15.69:0/3263549749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21e8004d40 0x7f21e8005150 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f21f8001050 tx=0x7f21e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcfe33bbf5f79b84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:18.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.752+0000 7f21fd59a640 1 -- 172.21.15.69:0/3263549749 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f800c0e0 con 0x7f21e8004d40 2024-04-25T10:59:18.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.752+0000 7f21fd59a640 1 -- 172.21.15.69:0/3263549749 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21f8034070 con 0x7f21e8004d40 2024-04-25T10:59:18.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.752+0000 7f21fd59a640 1 -- 172.21.15.69:0/3263549749 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f8005d10 con 0x7f21e8004d40 2024-04-25T10:59:18.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.753+0000 7f2204b01640 1 -- 172.21.15.69:0/3263549749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21e8004d40 msgr2=0x7f21e8005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:18.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.753+0000 7f2204b01640 1 --2- 172.21.15.69:0/3263549749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21e8004d40 0x7f21e8005150 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f21f8001050 tx=0x7f21e8005830 comp rx=0 tx=0).stop 2024-04-25T10:59:18.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.753+0000 7f21fed9d640 1 -- 172.21.15.69:0/3263549749 reap_dead start 2024-04-25T10:59:18.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.753+0000 7f2204b01640 1 -- 172.21.15.69:0/3263549749 shutdown_connections 2024-04-25T10:59:18.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.753+0000 7f2204b01640 1 -- 172.21.15.69:0/3263549749 >> 172.21.15.69:0/3263549749 conn(0x7f2200073090 msgr2=0x7f22000734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:18.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.753+0000 7f2204b01640 1 -- 172.21.15.69:0/3263549749 shutdown_connections 2024-04-25T10:59:18.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.753+0000 7f2204b01640 1 -- 172.21.15.69:0/3263549749 wait complete. 2024-04-25T10:59:18.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.755+0000 7f2204b01640 1 Processor -- start 2024-04-25T10:59:18.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.755+0000 7f2204b01640 1 -- start start 2024-04-25T10:59:18.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.756+0000 7f2204b01640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21e8004d40 0x7f2200138110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:18.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.756+0000 7f2204b01640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2200144670 0x7f2200144a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:18.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.756+0000 7f2204b01640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22000ac3b0 0x7f22000ac7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:18.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.756+0000 7f2204b01640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f22000b0f00 con 0x7f21e8004d40 2024-04-25T10:59:18.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.756+0000 7f2204b01640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f22000ce740 con 0x7f22000ac3b0 2024-04-25T10:59:18.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.756+0000 7f2204b01640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f22000cef40 con 0x7f2200144670 2024-04-25T10:59:18.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.756+0000 7f21fe59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21e8004d40 0x7f2200138110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:18.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.756+0000 7f21fe59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21e8004d40 0x7f2200138110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41066/0 (socket says 172.21.15.69:41066) 2024-04-25T10:59:18.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.756+0000 7f21fe59c640 1 -- 172.21.15.69:0/2750940026 learned_addr learned my addr 172.21.15.69:0/2750940026 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:18.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.756+0000 7f21fed9d640 1 --2- 172.21.15.69:0/2750940026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22000ac3b0 0x7f22000ac7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:18.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.756+0000 7f21fdd9b640 1 --2- 172.21.15.69:0/2750940026 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2200144670 0x7f2200144a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:18.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.757+0000 7f21fe59c640 1 -- 172.21.15.69:0/2750940026 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2200144670 msgr2=0x7f2200144a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:18.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.757+0000 7f21fe59c640 1 --2- 172.21.15.69:0/2750940026 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2200144670 0x7f2200144a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:18.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.757+0000 7f21fe59c640 1 -- 172.21.15.69:0/2750940026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22000ac3b0 msgr2=0x7f22000ac7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:18.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.757+0000 7f21fe59c640 1 --2- 172.21.15.69:0/2750940026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22000ac3b0 0x7f22000ac7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:18.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.757+0000 7f21fe59c640 1 -- 172.21.15.69:0/2750940026 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22000acd00 con 0x7f21e8004d40 2024-04-25T10:59:18.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.757+0000 7f21fed9d640 1 --2- 172.21.15.69:0/2750940026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22000ac3b0 0x7f22000ac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:18.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.757+0000 7f21fe59c640 1 --2- 172.21.15.69:0/2750940026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21e8004d40 0x7f2200138110 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f21f000f460 tx=0x7f21f000f860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:18.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.757+0000 7f21eeffd640 1 -- 172.21.15.69:0/2750940026 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f0003b90 con 0x7f21e8004d40 2024-04-25T10:59:18.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.757+0000 7f21eeffd640 1 -- 172.21.15.69:0/2750940026 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21f0003d50 con 0x7f21e8004d40 2024-04-25T10:59:18.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.757+0000 7f2204b01640 1 -- 172.21.15.69:0/2750940026 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2200145f20 con 0x7f21e8004d40 2024-04-25T10:59:18.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.757+0000 7f21eeffd640 1 -- 172.21.15.69:0/2750940026 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f0004270 con 0x7f21e8004d40 2024-04-25T10:59:18.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.757+0000 7f2204b01640 1 -- 172.21.15.69:0/2750940026 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2200145680 con 0x7f21e8004d40 2024-04-25T10:59:18.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.758+0000 7f2204b01640 1 -- 172.21.15.69:0/2750940026 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2200050740 con 0x7f21e8004d40 2024-04-25T10:59:18.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.758+0000 7f21eeffd640 1 -- 172.21.15.69:0/2750940026 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f21f0004410 con 0x7f21e8004d40 2024-04-25T10:59:18.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.759+0000 7f21eeffd640 1 --2- 172.21.15.69:0/2750940026 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f21e00744b0 0x7f21e00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:18.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.759+0000 7f21fdd9b640 1 --2- 172.21.15.69:0/2750940026 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f21e00744b0 0x7f21e00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:18.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.759+0000 7f21eeffd640 1 -- 172.21.15.69:0/2750940026 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f21f0096280 con 0x7f21e8004d40 2024-04-25T10:59:18.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.760+0000 7f21fdd9b640 1 --2- 172.21.15.69:0/2750940026 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f21e00744b0 0x7f21e00769c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f22000b2c10 tx=0x7f21f4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:18.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.764+0000 7f21eeffd640 1 -- 172.21.15.69:0/2750940026 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f22000acd00 con 0x7f21e8004d40 2024-04-25T10:59:18.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:18.974+0000 7f2204b01640 1 -- 172.21.15.69:0/2750940026 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f220015fcd0 con 0x7f21e8004d40 2024-04-25T10:59:19.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.616+0000 7f21eeffd640 1 -- 172.21.15.69:0/2750940026 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f21f0019070 con 0x7f21e8004d40 2024-04-25T10:59:19.617 INFO:teuthology.orchestra.run.smithi069.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T10:59:19.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.618+0000 7f2204b01640 1 -- 172.21.15.69:0/2750940026 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f21e00744b0 msgr2=0x7f21e00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:19.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.618+0000 7f2204b01640 1 --2- 172.21.15.69:0/2750940026 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f21e00744b0 0x7f21e00769c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f22000b2c10 tx=0x7f21f4009190 comp rx=0 tx=0).stop 2024-04-25T10:59:19.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.618+0000 7f2204b01640 1 -- 172.21.15.69:0/2750940026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21e8004d40 msgr2=0x7f2200138110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:19.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.618+0000 7f2204b01640 1 --2- 172.21.15.69:0/2750940026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21e8004d40 0x7f2200138110 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f21f000f460 tx=0x7f21f000f860 comp rx=0 tx=0).stop 2024-04-25T10:59:19.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.619+0000 7f2204b01640 1 -- 172.21.15.69:0/2750940026 shutdown_connections 2024-04-25T10:59:19.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.619+0000 7f2204b01640 1 --2- 172.21.15.69:0/2750940026 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f21e00744b0 0x7f21e00769c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:19.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.619+0000 7f2204b01640 1 --2- 172.21.15.69:0/2750940026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22000ac3b0 0x7f22000ac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:19.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.619+0000 7f2204b01640 1 --2- 172.21.15.69:0/2750940026 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2200144670 0x7f2200144a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:19.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.619+0000 7f2204b01640 1 --2- 172.21.15.69:0/2750940026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21e8004d40 0x7f2200138110 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:19.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.619+0000 7f2204b01640 1 -- 172.21.15.69:0/2750940026 >> 172.21.15.69:0/2750940026 conn(0x7f2200073090 msgr2=0x7f2200160510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:19.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.619+0000 7f2204b01640 1 -- 172.21.15.69:0/2750940026 shutdown_connections 2024-04-25T10:59:19.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.619+0000 7f2204b01640 1 -- 172.21.15.69:0/2750940026 wait complete. 2024-04-25T10:59:19.645 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:59:19.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.754+0000 7fedc0e4c640 1 Processor -- start 2024-04-25T10:59:19.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.754+0000 7fedc0e4c640 1 -- start start 2024-04-25T10:59:19.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.755+0000 7fedc0e4c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fedbc0b8290 0x7fedbc160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:19.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.756+0000 7fedc0e4c640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedbc1615f0 con 0x7fedbc164b00 2024-04-25T10:59:19.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.756+0000 7fedc0e4c640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedbc164020 con 0x7fedbc15df80 2024-04-25T10:59:19.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.756+0000 7fedc0e4c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fedbc164670 con 0x7fedbc0b8290 2024-04-25T10:59:19.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.756+0000 7fedba59c640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fedbc164b00 0x7fedbc0b7b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:57104/0 (socket says 172.21.15.69:57104) 2024-04-25T10:59:19.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.756+0000 7fedba59c640 1 -- 172.21.15.69:0/1936305267 learned_addr learned my addr 172.21.15.69:0/1936305267 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:19.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.756+0000 7fedb9d9b640 1 --2- 172.21.15.69:0/1936305267 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fedbc0b8290 0x7fedbc160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:19.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.756+0000 7fedb959a640 1 -- 172.21.15.69:0/1936305267 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2842493411 0 0) 0x7fedbc1615f0 con 0x7fedbc164b00 2024-04-25T10:59:19.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.756+0000 7fedb959a640 1 -- 172.21.15.69:0/1936305267 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feda4003520 con 0x7fedbc164b00 2024-04-25T10:59:19.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.756+0000 7fedb959a640 1 -- 172.21.15.69:0/1936305267 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 281430853 0 0) 0x7feda4003520 con 0x7fedbc164b00 2024-04-25T10:59:19.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.756+0000 7fedb959a640 1 -- 172.21.15.69:0/1936305267 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fedbc0b8290 msgr2=0x7fedbc160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:19.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.756+0000 7fedb959a640 1 --2- 172.21.15.69:0/1936305267 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fedbc0b8290 0x7fedbc160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:19.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.756+0000 7fedb959a640 1 -- 172.21.15.69:0/1936305267 >> v1:172.21.15.146:6789/0 conn(0x7fedbc15df80 legacy=0x7fedbc160dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:19.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.756+0000 7fedb959a640 1 -- 172.21.15.69:0/1936305267 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedbc164300 con 0x7fedbc164b00 2024-04-25T10:59:19.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.756+0000 7fedb959a640 1 -- 172.21.15.69:0/1936305267 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7fedac0019f0 con 0x7fedbc164b00 2024-04-25T10:59:19.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.756+0000 7fedb959a640 1 -- 172.21.15.69:0/1936305267 >> v1:172.21.15.69:6789/0 conn(0x7fedbc164b00 legacy=0x7fedbc0b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:19.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.756+0000 7fedb959a640 1 --2- 172.21.15.69:0/1936305267 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feda40039e0 0x7feda4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:19.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.756+0000 7fedb959a640 1 --2- 172.21.15.69:0/1936305267 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feda4004800 0x7feda4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:19.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.756+0000 7fedb959a640 1 --2- 172.21.15.69:0/1936305267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feda4005150 0x7feda400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:19.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.756+0000 7fedb959a640 1 -- 172.21.15.69:0/1936305267 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fedbc164670 con 0x7feda4005150 2024-04-25T10:59:19.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.756+0000 7fedb959a640 1 -- 172.21.15.69:0/1936305267 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7feda400bf00 con 0x7feda40039e0 2024-04-25T10:59:19.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.756+0000 7fedb959a640 1 -- 172.21.15.69:0/1936305267 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7feda400c100 con 0x7feda4004800 2024-04-25T10:59:19.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.756+0000 7fedbad9d640 1 --2- 172.21.15.69:0/1936305267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feda4005150 0x7feda400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:19.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.756+0000 7fedba59c640 1 --2- 172.21.15.69:0/1936305267 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feda40039e0 0x7feda4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:19.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.756+0000 7fedb9d9b640 1 --2- 172.21.15.69:0/1936305267 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feda4004800 0x7feda4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:19.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.757+0000 7fedbad9d640 1 -- 172.21.15.69:0/1936305267 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feda4004800 msgr2=0x7feda4004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:19.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.757+0000 7fedbad9d640 1 --2- 172.21.15.69:0/1936305267 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feda4004800 0x7feda4004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:19.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.757+0000 7fedbad9d640 1 -- 172.21.15.69:0/1936305267 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feda40039e0 msgr2=0x7feda4003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:19.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.757+0000 7fedbad9d640 1 --2- 172.21.15.69:0/1936305267 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feda40039e0 0x7feda4003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:19.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.757+0000 7fedbad9d640 1 -- 172.21.15.69:0/1936305267 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda4003520 con 0x7feda4005150 2024-04-25T10:59:19.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.757+0000 7fedbad9d640 1 --2- 172.21.15.69:0/1936305267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feda4005150 0x7feda400b9c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fedbc0b87c0 tx=0x7feda4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60da2af8a30f40ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:19.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.757+0000 7fedb959a640 1 -- 172.21.15.69:0/1936305267 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb4001370 con 0x7feda4005150 2024-04-25T10:59:19.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.757+0000 7fedb959a640 1 -- 172.21.15.69:0/1936305267 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedb40052d0 con 0x7feda4005150 2024-04-25T10:59:19.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.757+0000 7fedb959a640 1 -- 172.21.15.69:0/1936305267 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb402ab90 con 0x7feda4005150 2024-04-25T10:59:19.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.758+0000 7fedc0e4c640 1 -- 172.21.15.69:0/1936305267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feda4005150 msgr2=0x7feda400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:19.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.758+0000 7fedc0e4c640 1 --2- 172.21.15.69:0/1936305267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feda4005150 0x7feda400b9c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fedbc0b87c0 tx=0x7feda4005830 comp rx=0 tx=0).stop 2024-04-25T10:59:19.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.758+0000 7fedbad9d640 1 -- 172.21.15.69:0/1936305267 reap_dead start 2024-04-25T10:59:19.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.758+0000 7fedc0e4c640 1 -- 172.21.15.69:0/1936305267 shutdown_connections 2024-04-25T10:59:19.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.758+0000 7fedc0e4c640 1 -- 172.21.15.69:0/1936305267 >> 172.21.15.69:0/1936305267 conn(0x7fedbc0d2bc0 msgr2=0x7fedbc0d2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:19.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.758+0000 7fedc0e4c640 1 -- 172.21.15.69:0/1936305267 shutdown_connections 2024-04-25T10:59:19.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.758+0000 7fedc0e4c640 1 -- 172.21.15.69:0/1936305267 wait complete. 2024-04-25T10:59:19.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.760+0000 7fedc0e4c640 1 Processor -- start 2024-04-25T10:59:19.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.760+0000 7fedc0e4c640 1 -- start start 2024-04-25T10:59:19.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.761+0000 7fedc0e4c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feda4005150 0x7fedbc0ac700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:19.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.761+0000 7fedc0e4c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedbc0cca90 0x7fedbc0ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:19.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.761+0000 7fedc0e4c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedbc0cd3e0 0x7fedbc1f5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:19.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.761+0000 7fedc0e4c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fedbc160890 con 0x7fedbc0cca90 2024-04-25T10:59:19.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.761+0000 7fedc0e4c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fedbc160a10 con 0x7fedbc0cd3e0 2024-04-25T10:59:19.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.761+0000 7fedc0e4c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fedbc160b90 con 0x7feda4005150 2024-04-25T10:59:19.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.762+0000 7fedb9d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedbc0cca90 0x7fedbc0ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:19.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.762+0000 7fedb9d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedbc0cca90 0x7fedbc0ccea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41084/0 (socket says 172.21.15.69:41084) 2024-04-25T10:59:19.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.762+0000 7fedb9d9b640 1 -- 172.21.15.69:0/3448521391 learned_addr learned my addr 172.21.15.69:0/3448521391 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:19.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.762+0000 7fedba59c640 1 --2- 172.21.15.69:0/3448521391 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feda4005150 0x7fedbc0ac700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:19.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.762+0000 7fedbad9d640 1 --2- 172.21.15.69:0/3448521391 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedbc0cd3e0 0x7fedbc1f5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:19.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.762+0000 7fedb9d9b640 1 -- 172.21.15.69:0/3448521391 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feda4005150 msgr2=0x7fedbc0ac700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:19.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.762+0000 7fedb9d9b640 1 --2- 172.21.15.69:0/3448521391 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feda4005150 0x7fedbc0ac700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:19.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.762+0000 7fedb9d9b640 1 -- 172.21.15.69:0/3448521391 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedbc0cd3e0 msgr2=0x7fedbc1f5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:19.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.762+0000 7fedb9d9b640 1 --2- 172.21.15.69:0/3448521391 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedbc0cd3e0 0x7fedbc1f5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:19.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.762+0000 7fedb9d9b640 1 -- 172.21.15.69:0/3448521391 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedbc15d680 con 0x7fedbc0cca90 2024-04-25T10:59:19.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.762+0000 7fedb9d9b640 1 --2- 172.21.15.69:0/3448521391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedbc0cca90 0x7fedbc0ccea0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fedb00119a0 tx=0x7fedb0011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:19.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.762+0000 7fedaaffd640 1 -- 172.21.15.69:0/3448521391 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb000faf0 con 0x7fedbc0cca90 2024-04-25T10:59:19.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.762+0000 7fedaaffd640 1 -- 172.21.15.69:0/3448521391 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedb000fc90 con 0x7fedbc0cca90 2024-04-25T10:59:19.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.762+0000 7fedaaffd640 1 -- 172.21.15.69:0/3448521391 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb0014440 con 0x7fedbc0cca90 2024-04-25T10:59:19.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.762+0000 7fedc0e4c640 1 -- 172.21.15.69:0/3448521391 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedbc0ad1a0 con 0x7fedbc0cca90 2024-04-25T10:59:19.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.762+0000 7fedc0e4c640 1 -- 172.21.15.69:0/3448521391 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedbc0ad760 con 0x7fedbc0cca90 2024-04-25T10:59:19.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.763+0000 7fedc0e4c640 1 -- 172.21.15.69:0/3448521391 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedbc050740 con 0x7fedbc0cca90 2024-04-25T10:59:19.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.764+0000 7fedaaffd640 1 -- 172.21.15.69:0/3448521391 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fedb0012040 con 0x7fedbc0cca90 2024-04-25T10:59:19.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.764+0000 7fedaaffd640 1 --2- 172.21.15.69:0/3448521391 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fed9c0744b0 0x7fed9c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:19.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.764+0000 7fedba59c640 1 --2- 172.21.15.69:0/3448521391 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fed9c0744b0 0x7fed9c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:19.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.764+0000 7fedaaffd640 1 -- 172.21.15.69:0/3448521391 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fedb00958d0 con 0x7fedbc0cca90 2024-04-25T10:59:19.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.765+0000 7fedba59c640 1 --2- 172.21.15.69:0/3448521391 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fed9c0744b0 0x7fed9c0769c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fedac0013d0 tx=0x7fedac006040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:19.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.769+0000 7fedaaffd640 1 -- 172.21.15.69:0/3448521391 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fedb0019030 con 0x7fedbc0cca90 2024-04-25T10:59:19.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.947+0000 7fedc0e4c640 1 -- 172.21.15.69:0/3448521391 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fedbc0d4170 con 0x7fedbc0cca90 2024-04-25T10:59:19.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.949+0000 7fedaaffd640 1 -- 172.21.15.69:0/3448521391 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16618 (secure 0 0 0) 0x7fedb005f8f0 con 0x7fedbc0cca90 2024-04-25T10:59:19.950 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:19.951 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":19,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T10:59:18.976274+0000","last_up_change":"2024-04-25T10:59:05.546839+0000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:59:06.422674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:59:16.727980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:59:17.938486+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6825","nonce":1876101065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6827","nonce":1876101065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6831","nonce":1876101065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6829","nonce":1876101065}]},"public_addr":"172.21.15.69:6825/1876101065","cluster_addr":"172.21.15.69:6827/1876101065","heartbeat_back_addr":"172.21.15.69:6831/1876101065","heartbeat_front_addr":"172.21.15.69:6829/1876101065","state":["exists","up"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6801","nonce":1806438563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6803","nonce":1806438563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6807","nonce":1806438563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6805","nonce":1806438563}]},"public_addr":"172.21.15.69:6801/1806438563","cluster_addr":"172.21.15.69:6803/1806438563","heartbeat_back_addr":"172.21.15.69:6807/1806438563","heartbeat_front_addr":"172.21.15.69:6805/1806438563","state":["exists","up"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6817","nonce":1593617983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6819","nonce":1593617983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6823","nonce":1593617983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6821","nonce":1593617983}]},"public_addr":"172.21.15.69:6817/1593617983","cluster_addr":"172.21.15.69:6819/1593617983","heartbeat_back_addr":"172.21.15.69:6823/1593617983","heartbeat_front_addr":"172.21.15.69:6821/1593617983","state":["exists","up"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6809","nonce":3950661294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6811","nonce":3950661294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6815","nonce":3950661294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6813","nonce":3950661294}]},"public_addr":"172.21.15.69:6809/3950661294","cluster_addr":"172.21.15.69:6811/3950661294","heartbeat_back_addr":"172.21.15.69:6815/3950661294","heartbeat_front_addr":"172.21.15.69:6813/3950661294","state":["exists","up"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6801","nonce":2508929872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6803","nonce":2508929872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6807","nonce":2508929872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6805","nonce":2508929872}]},"public_addr":"172.21.15.146:6801/2508929872","cluster_addr":"172.21.15.146:6803/2508929872","heartbeat_back_addr":"172.21.15.146:6807/2508929872","heartbeat_front_addr":"172.21.15.146:6805/2508929872","state":["exists","up"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6811","nonce":1310911138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6816","nonce":1310911138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6823","nonce":1310911138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6821","nonce":1310911138}]},"public_addr":"172.21.15.146:6811/1310911138","cluster_addr":"172.21.15.146:6816/1310911138","heartbeat_back_addr":"172.21.15.146:6823/1310911138","heartbeat_front_addr":"172.21.15.146:6821/1310911138","state":["exists","up"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6810","nonce":634688021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6813","nonce":634688021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6820","nonce":634688021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6817","nonce":634688021}]},"public_addr":"172.21.15.146:6810/634688021","cluster_addr":"172.21.15.146:6813/634688021","heartbeat_back_addr":"172.21.15.146:6820/634688021","heartbeat_front_addr":"172.21.15.146:6817/634688021","state":["exists","up"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6825","nonce":3814215935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6827","nonce":3814215935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6831","nonce":3814215935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6829","nonce":3814215935}]},"public_addr":"172.21.15.146:6825/3814215935","cluster_addr":"172.21.15.146:6827/3814215935","heartbeat_back_addr":"172.21.15.146:6831/3814215935","heartbeat_front_addr":"172.21.15.146:6829/3814215935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.820200+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.850189+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.764560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.866248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.807897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.869881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.782310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.827131+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:59:19.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.951+0000 7fedc0e4c640 1 -- 172.21.15.69:0/3448521391 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fed9c0744b0 msgr2=0x7fed9c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:19.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.951+0000 7fedc0e4c640 1 --2- 172.21.15.69:0/3448521391 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fed9c0744b0 0x7fed9c0769c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fedac0013d0 tx=0x7fedac006040 comp rx=0 tx=0).stop 2024-04-25T10:59:19.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.951+0000 7fedc0e4c640 1 -- 172.21.15.69:0/3448521391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedbc0cca90 msgr2=0x7fedbc0ccea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:19.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.951+0000 7fedc0e4c640 1 --2- 172.21.15.69:0/3448521391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedbc0cca90 0x7fedbc0ccea0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fedb00119a0 tx=0x7fedb0011e60 comp rx=0 tx=0).stop 2024-04-25T10:59:19.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.951+0000 7fedc0e4c640 1 -- 172.21.15.69:0/3448521391 shutdown_connections 2024-04-25T10:59:19.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.951+0000 7fedc0e4c640 1 --2- 172.21.15.69:0/3448521391 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fed9c0744b0 0x7fed9c0769c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:19.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.951+0000 7fedc0e4c640 1 --2- 172.21.15.69:0/3448521391 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedbc0cd3e0 0x7fedbc1f5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:19.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.951+0000 7fedc0e4c640 1 --2- 172.21.15.69:0/3448521391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedbc0cca90 0x7fedbc0ccea0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:19.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.951+0000 7fedc0e4c640 1 --2- 172.21.15.69:0/3448521391 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feda4005150 0x7fedbc0ac700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:19.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.951+0000 7fedc0e4c640 1 -- 172.21.15.69:0/3448521391 >> 172.21.15.69:0/3448521391 conn(0x7fedbc0d2bc0 msgr2=0x7fedbc15efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:19.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.951+0000 7fedc0e4c640 1 -- 172.21.15.69:0/3448521391 shutdown_connections 2024-04-25T10:59:19.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:19.951+0000 7fedc0e4c640 1 -- 172.21.15.69:0/3448521391 wait complete. 2024-04-25T10:59:19.963 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:59:20.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.070+0000 7f82fa3be640 1 Processor -- start 2024-04-25T10:59:20.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.070+0000 7f82fa3be640 1 -- start start 2024-04-25T10:59:20.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.072+0000 7f82fa3be640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f82f40d1210 0x7f82f40d5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:20.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.072+0000 7f82fa3be640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82f40d1850 con 0x7f82f406ccd0 2024-04-25T10:59:20.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.072+0000 7f82fa3be640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82f4160de0 con 0x7f82f40d0e00 2024-04-25T10:59:20.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.072+0000 7f82fa3be640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f82f40d1620 con 0x7f82f40d1210 2024-04-25T10:59:20.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.072+0000 7f82f37fe640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f82f406ccd0 0x7f82f40d4a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:57110/0 (socket says 172.21.15.69:57110) 2024-04-25T10:59:20.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.072+0000 7f82f37fe640 1 -- 172.21.15.69:0/2096101013 learned_addr learned my addr 172.21.15.69:0/2096101013 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:20.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.072+0000 7f82f895b640 1 --2- 172.21.15.69:0/2096101013 >> v2:172.21.15.146:3300/0 conn(0x7f82f40d1210 0x7f82f40d5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:20.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.072+0000 7f82f2ffd640 1 -- 172.21.15.69:0/2096101013 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 849694109 0 0) 0x7f82f40d1850 con 0x7f82f406ccd0 2024-04-25T10:59:20.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.072+0000 7f82f2ffd640 1 -- 172.21.15.69:0/2096101013 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f82dc003520 con 0x7f82f406ccd0 2024-04-25T10:59:20.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.072+0000 7f82f2ffd640 1 -- 172.21.15.69:0/2096101013 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 902798786 0 0) 0x7f82f4160de0 con 0x7f82f40d0e00 2024-04-25T10:59:20.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.072+0000 7f82f2ffd640 1 -- 172.21.15.69:0/2096101013 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f82f40d1850 con 0x7f82f40d0e00 2024-04-25T10:59:20.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.072+0000 7f82f2ffd640 1 -- 172.21.15.69:0/2096101013 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 483573209 0 0) 0x7f82dc003520 con 0x7f82f406ccd0 2024-04-25T10:59:20.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.072+0000 7f82f2ffd640 1 -- 172.21.15.69:0/2096101013 >> v2:172.21.15.146:3300/0 conn(0x7f82f40d1210 msgr2=0x7f82f40d5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:20.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.072+0000 7f82f2ffd640 1 --2- 172.21.15.69:0/2096101013 >> v2:172.21.15.146:3300/0 conn(0x7f82f40d1210 0x7f82f40d5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:20.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.072+0000 7f82f2ffd640 1 -- 172.21.15.69:0/2096101013 >> v1:172.21.15.146:6789/0 conn(0x7f82f40d0e00 legacy=0x7f82f406c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:20.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.072+0000 7f82f2ffd640 1 -- 172.21.15.69:0/2096101013 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f41610c0 con 0x7f82f406ccd0 2024-04-25T10:59:20.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.072+0000 7f82f2ffd640 1 -- 172.21.15.69:0/2096101013 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7f82e8001670 con 0x7f82f406ccd0 2024-04-25T10:59:20.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.072+0000 7f82f2ffd640 1 -- 172.21.15.69:0/2096101013 >> v1:172.21.15.69:6789/0 conn(0x7f82f406ccd0 legacy=0x7f82f40d4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:20.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.072+0000 7f82f2ffd640 1 --2- 172.21.15.69:0/2096101013 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82dc003b40 0x7f82dc003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:20.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.072+0000 7f82f2ffd640 1 --2- 172.21.15.69:0/2096101013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82dc004800 0x7f82dc004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:20.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.072+0000 7f82f2ffd640 1 --2- 172.21.15.69:0/2096101013 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82dc005260 0x7f82dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:20.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.072+0000 7f82f2ffd640 1 -- 172.21.15.69:0/2096101013 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc00c190 con 0x7f82dc004800 2024-04-25T10:59:20.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.072+0000 7f82f2ffd640 1 -- 172.21.15.69:0/2096101013 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc00c340 con 0x7f82dc003b40 2024-04-25T10:59:20.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.073+0000 7f82f2ffd640 1 -- 172.21.15.69:0/2096101013 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f82dc00c540 con 0x7f82dc005260 2024-04-25T10:59:20.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.073+0000 7f82f895b640 1 --2- 172.21.15.69:0/2096101013 >> v2:172.21.15.146:3300/0 conn(0x7f82f40d1210 0x7f82f40d5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:59:20.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.073+0000 7f82f37fe640 1 --2- 172.21.15.69:0/2096101013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82dc004800 0x7f82dc004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:20.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.073+0000 7f82f3fff640 1 --2- 172.21.15.69:0/2096101013 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82dc003b40 0x7f82dc003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:20.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.073+0000 7f82f895b640 1 --2- 172.21.15.69:0/2096101013 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82dc005260 0x7f82dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:20.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.073+0000 7f82f37fe640 1 -- 172.21.15.69:0/2096101013 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82dc005260 msgr2=0x7f82dc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:20.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.073+0000 7f82f37fe640 1 --2- 172.21.15.69:0/2096101013 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82dc005260 0x7f82dc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:20.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.073+0000 7f82f37fe640 1 -- 172.21.15.69:0/2096101013 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82dc003b40 msgr2=0x7f82dc003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:20.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.073+0000 7f82f37fe640 1 --2- 172.21.15.69:0/2096101013 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82dc003b40 0x7f82dc003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:20.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.073+0000 7f82f37fe640 1 -- 172.21.15.69:0/2096101013 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82dc003520 con 0x7f82dc004800 2024-04-25T10:59:20.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.073+0000 7f82f895b640 1 --2- 172.21.15.69:0/2096101013 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82dc005260 0x7f82dc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:20.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.073+0000 7f82f895b640 1 -- 172.21.15.69:0/2096101013 reap_dead start 2024-04-25T10:59:20.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.074+0000 7f82f37fe640 1 --2- 172.21.15.69:0/2096101013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82dc004800 0x7f82dc004c10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f82e8007fd0 tx=0x7f82dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=594cf74a32cbfd3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:20.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.074+0000 7f82f2ffd640 1 -- 172.21.15.69:0/2096101013 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e8007ca0 con 0x7f82dc004800 2024-04-25T10:59:20.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.074+0000 7f82f2ffd640 1 -- 172.21.15.69:0/2096101013 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82e8005af0 con 0x7f82dc004800 2024-04-25T10:59:20.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.074+0000 7f82f2ffd640 1 -- 172.21.15.69:0/2096101013 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e8007070 con 0x7f82dc004800 2024-04-25T10:59:20.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.074+0000 7f82fa3be640 1 -- 172.21.15.69:0/2096101013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82dc004800 msgr2=0x7f82dc004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:20.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.074+0000 7f82fa3be640 1 --2- 172.21.15.69:0/2096101013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82dc004800 0x7f82dc004c10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f82e8007fd0 tx=0x7f82dc005830 comp rx=0 tx=0).stop 2024-04-25T10:59:20.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.074+0000 7f82fa3be640 1 -- 172.21.15.69:0/2096101013 shutdown_connections 2024-04-25T10:59:20.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.074+0000 7f82fa3be640 1 --2- 172.21.15.69:0/2096101013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82dc004800 0x7f82dc004c10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:20.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.074+0000 7f82fa3be640 1 -- 172.21.15.69:0/2096101013 >> 172.21.15.69:0/2096101013 conn(0x7f82f40d5d90 msgr2=0x7f82f40d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:20.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.074+0000 7f82fa3be640 1 -- 172.21.15.69:0/2096101013 shutdown_connections 2024-04-25T10:59:20.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.074+0000 7f82fa3be640 1 -- 172.21.15.69:0/2096101013 wait complete. 2024-04-25T10:59:20.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.076+0000 7f82fa3be640 1 Processor -- start 2024-04-25T10:59:20.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.076+0000 7f82fa3be640 1 -- start start 2024-04-25T10:59:20.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.077+0000 7f82fa3be640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82dc004800 0x7f82f415ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:20.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.078+0000 7f82fa3be640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82f414e810 0x7f82f414dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:20.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.078+0000 7f82fa3be640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82f414bbd0 0x7f82f414bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:20.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.078+0000 7f82fa3be640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f82f40b5320 con 0x7f82f414bbd0 2024-04-25T10:59:20.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.078+0000 7f82fa3be640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f82f4160ad0 con 0x7f82dc004800 2024-04-25T10:59:20.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.078+0000 7f82fa3be640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f82f4164e00 con 0x7f82f414e810 2024-04-25T10:59:20.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.078+0000 7f82f895b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82f414bbd0 0x7f82f414bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:20.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.078+0000 7f82f37fe640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82f414e810 0x7f82f414dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:20.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.078+0000 7f82f895b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82f414bbd0 0x7f82f414bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41108/0 (socket says 172.21.15.69:41108) 2024-04-25T10:59:20.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.078+0000 7f82f895b640 1 -- 172.21.15.69:0/2236283904 learned_addr learned my addr 172.21.15.69:0/2236283904 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:20.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.078+0000 7f82f3fff640 1 --2- 172.21.15.69:0/2236283904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82dc004800 0x7f82f415ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:20.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.078+0000 7f82f895b640 1 -- 172.21.15.69:0/2236283904 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82f414e810 msgr2=0x7f82f414dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:20.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.078+0000 7f82f895b640 1 --2- 172.21.15.69:0/2236283904 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82f414e810 0x7f82f414dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:20.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.078+0000 7f82f895b640 1 -- 172.21.15.69:0/2236283904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82dc004800 msgr2=0x7f82f415ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:20.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.078+0000 7f82f895b640 1 --2- 172.21.15.69:0/2236283904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82dc004800 0x7f82f415ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:20.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.078+0000 7f82f895b640 1 -- 172.21.15.69:0/2236283904 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f415a9c0 con 0x7f82f414bbd0 2024-04-25T10:59:20.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.078+0000 7f82f37fe640 1 --2- 172.21.15.69:0/2236283904 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82f414e810 0x7f82f414dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:59:20.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.078+0000 7f82f895b640 1 --2- 172.21.15.69:0/2236283904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82f414bbd0 0x7f82f414bfe0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f82f40d4050 tx=0x7f82ec0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:20.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.078+0000 7f82f0ff9640 1 -- 172.21.15.69:0/2236283904 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec003990 con 0x7f82f414bbd0 2024-04-25T10:59:20.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.078+0000 7f82f0ff9640 1 -- 172.21.15.69:0/2236283904 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82ec01b070 con 0x7f82f414bbd0 2024-04-25T10:59:20.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.078+0000 7f82f0ff9640 1 -- 172.21.15.69:0/2236283904 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec003b90 con 0x7f82f414bbd0 2024-04-25T10:59:20.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.078+0000 7f82fa3be640 1 -- 172.21.15.69:0/2236283904 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f415aca0 con 0x7f82f414bbd0 2024-04-25T10:59:20.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.078+0000 7f82fa3be640 1 -- 172.21.15.69:0/2236283904 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f415b260 con 0x7f82f414bbd0 2024-04-25T10:59:20.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.079+0000 7f82fa3be640 1 -- 172.21.15.69:0/2236283904 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82f40d6410 con 0x7f82f414bbd0 2024-04-25T10:59:20.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.080+0000 7f82f0ff9640 1 -- 172.21.15.69:0/2236283904 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f82ec005d10 con 0x7f82f414bbd0 2024-04-25T10:59:20.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.080+0000 7f82f0ff9640 1 --2- 172.21.15.69:0/2236283904 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f82e00744b0 0x7f82e00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:20.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.081+0000 7f82f3fff640 1 --2- 172.21.15.69:0/2236283904 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f82e00744b0 0x7f82e00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:20.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.081+0000 7f82f0ff9640 1 -- 172.21.15.69:0/2236283904 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f82ec0957e0 con 0x7f82f414bbd0 2024-04-25T10:59:20.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.081+0000 7f82f3fff640 1 --2- 172.21.15.69:0/2236283904 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f82e00744b0 0x7f82e00769c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f82e4000c00 tx=0x7f82e4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:20.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.085+0000 7f82f0ff9640 1 -- 172.21.15.69:0/2236283904 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f82f40d6410 con 0x7f82f414bbd0 2024-04-25T10:59:20.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.263+0000 7f82fa3be640 1 -- 172.21.15.69:0/2236283904 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82f415a750 con 0x7f82f414bbd0 2024-04-25T10:59:20.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.264+0000 7f82f0ff9640 1 -- 172.21.15.69:0/2236283904 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16618 (secure 0 0 0) 0x7f82ec025020 con 0x7f82f414bbd0 2024-04-25T10:59:20.265 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:20.266 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":19,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T10:59:18.976274+0000","last_up_change":"2024-04-25T10:59:05.546839+0000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:59:06.422674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:59:16.727980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:59:17.938486+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6825","nonce":1876101065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6827","nonce":1876101065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6831","nonce":1876101065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6829","nonce":1876101065}]},"public_addr":"172.21.15.69:6825/1876101065","cluster_addr":"172.21.15.69:6827/1876101065","heartbeat_back_addr":"172.21.15.69:6831/1876101065","heartbeat_front_addr":"172.21.15.69:6829/1876101065","state":["exists","up"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6801","nonce":1806438563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6803","nonce":1806438563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6807","nonce":1806438563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6805","nonce":1806438563}]},"public_addr":"172.21.15.69:6801/1806438563","cluster_addr":"172.21.15.69:6803/1806438563","heartbeat_back_addr":"172.21.15.69:6807/1806438563","heartbeat_front_addr":"172.21.15.69:6805/1806438563","state":["exists","up"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6817","nonce":1593617983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6819","nonce":1593617983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6823","nonce":1593617983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6821","nonce":1593617983}]},"public_addr":"172.21.15.69:6817/1593617983","cluster_addr":"172.21.15.69:6819/1593617983","heartbeat_back_addr":"172.21.15.69:6823/1593617983","heartbeat_front_addr":"172.21.15.69:6821/1593617983","state":["exists","up"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6809","nonce":3950661294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6811","nonce":3950661294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6815","nonce":3950661294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6813","nonce":3950661294}]},"public_addr":"172.21.15.69:6809/3950661294","cluster_addr":"172.21.15.69:6811/3950661294","heartbeat_back_addr":"172.21.15.69:6815/3950661294","heartbeat_front_addr":"172.21.15.69:6813/3950661294","state":["exists","up"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6801","nonce":2508929872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6803","nonce":2508929872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6807","nonce":2508929872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6805","nonce":2508929872}]},"public_addr":"172.21.15.146:6801/2508929872","cluster_addr":"172.21.15.146:6803/2508929872","heartbeat_back_addr":"172.21.15.146:6807/2508929872","heartbeat_front_addr":"172.21.15.146:6805/2508929872","state":["exists","up"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6811","nonce":1310911138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6816","nonce":1310911138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6823","nonce":1310911138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6821","nonce":1310911138}]},"public_addr":"172.21.15.146:6811/1310911138","cluster_addr":"172.21.15.146:6816/1310911138","heartbeat_back_addr":"172.21.15.146:6823/1310911138","heartbeat_front_addr":"172.21.15.146:6821/1310911138","state":["exists","up"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6810","nonce":634688021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6813","nonce":634688021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6820","nonce":634688021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6817","nonce":634688021}]},"public_addr":"172.21.15.146:6810/634688021","cluster_addr":"172.21.15.146:6813/634688021","heartbeat_back_addr":"172.21.15.146:6820/634688021","heartbeat_front_addr":"172.21.15.146:6817/634688021","state":["exists","up"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6825","nonce":3814215935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6827","nonce":3814215935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6831","nonce":3814215935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6829","nonce":3814215935}]},"public_addr":"172.21.15.146:6825/3814215935","cluster_addr":"172.21.15.146:6827/3814215935","heartbeat_back_addr":"172.21.15.146:6831/3814215935","heartbeat_front_addr":"172.21.15.146:6829/3814215935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.820200+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.850189+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.764560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.866248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.807897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.869881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.782310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.827131+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:59:20.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.266+0000 7f82fa3be640 1 -- 172.21.15.69:0/2236283904 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f82e00744b0 msgr2=0x7f82e00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:20.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.266+0000 7f82fa3be640 1 --2- 172.21.15.69:0/2236283904 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f82e00744b0 0x7f82e00769c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f82e4000c00 tx=0x7f82e4009040 comp rx=0 tx=0).stop 2024-04-25T10:59:20.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.266+0000 7f82fa3be640 1 -- 172.21.15.69:0/2236283904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82f414bbd0 msgr2=0x7f82f414bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:20.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.266+0000 7f82fa3be640 1 --2- 172.21.15.69:0/2236283904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82f414bbd0 0x7f82f414bfe0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f82f40d4050 tx=0x7f82ec0115a0 comp rx=0 tx=0).stop 2024-04-25T10:59:20.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.266+0000 7f82fa3be640 1 -- 172.21.15.69:0/2236283904 shutdown_connections 2024-04-25T10:59:20.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.266+0000 7f82fa3be640 1 --2- 172.21.15.69:0/2236283904 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f82e00744b0 0x7f82e00769c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:20.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.266+0000 7f82fa3be640 1 --2- 172.21.15.69:0/2236283904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82f414bbd0 0x7f82f414bfe0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:20.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.266+0000 7f82fa3be640 1 --2- 172.21.15.69:0/2236283904 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82f414e810 0x7f82f414dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:20.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.266+0000 7f82fa3be640 1 --2- 172.21.15.69:0/2236283904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82dc004800 0x7f82f415ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:20.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.266+0000 7f82fa3be640 1 -- 172.21.15.69:0/2236283904 >> 172.21.15.69:0/2236283904 conn(0x7f82f40d5d90 msgr2=0x7f82f415ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:20.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.267+0000 7f82fa3be640 1 -- 172.21.15.69:0/2236283904 shutdown_connections 2024-04-25T10:59:20.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.267+0000 7f82fa3be640 1 -- 172.21.15.69:0/2236283904 wait complete. 2024-04-25T10:59:20.279 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:59:20.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.388+0000 7fb0210e7640 1 Processor -- start 2024-04-25T10:59:20.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.388+0000 7fb0210e7640 1 -- start start 2024-04-25T10:59:20.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.389+0000 7fb0210e7640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb01c161710 0x7fb01c15e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:20.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.389+0000 7fb0210e7640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb01c1609b0 con 0x7fb01c163fa0 2024-04-25T10:59:20.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.389+0000 7fb0210e7640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb01c164cd0 con 0x7fb01c06c5c0 2024-04-25T10:59:20.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.389+0000 7fb0210e7640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fb01c160c30 con 0x7fb01c161710 2024-04-25T10:59:20.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.389+0000 7fb01b59e640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fb01c163fa0 0x7fb01c1643b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:57124/0 (socket says 172.21.15.69:57124) 2024-04-25T10:59:20.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.389+0000 7fb01b59e640 1 -- 172.21.15.69:0/207955948 learned_addr learned my addr 172.21.15.69:0/207955948 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:20.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.389+0000 7fb01ad9d640 1 --2- 172.21.15.69:0/207955948 >> v2:172.21.15.146:3300/0 conn(0x7fb01c161710 0x7fb01c15e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:20.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.390+0000 7fb01a59c640 1 -- 172.21.15.69:0/207955948 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 767109622 0 0) 0x7fb014001490 con 0x7fb01c163fa0 2024-04-25T10:59:20.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.390+0000 7fb01a59c640 1 -- 172.21.15.69:0/207955948 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb008003520 con 0x7fb01c163fa0 2024-04-25T10:59:20.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.390+0000 7fb01a59c640 1 -- 172.21.15.69:0/207955948 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2833166774 0 0) 0x7fb01c164cd0 con 0x7fb01c06c5c0 2024-04-25T10:59:20.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.390+0000 7fb01a59c640 1 -- 172.21.15.69:0/207955948 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb014001490 con 0x7fb01c06c5c0 2024-04-25T10:59:20.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.390+0000 7fb01ad9d640 1 -- 172.21.15.69:0/207955948 >> v1:172.21.15.146:6789/0 conn(0x7fb01c06c5c0 legacy=0x7fb01c06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:20.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.390+0000 7fb01ad9d640 1 -- 172.21.15.69:0/207955948 >> v1:172.21.15.69:6789/0 conn(0x7fb01c163fa0 legacy=0x7fb01c1643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:20.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.390+0000 7fb01ad9d640 1 -- 172.21.15.69:0/207955948 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01c160f10 con 0x7fb01c161710 2024-04-25T10:59:20.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.391+0000 7fb01ad9d640 1 --2- 172.21.15.69:0/207955948 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb01c161710 0x7fb01c15e080 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0097a0 tx=0x7fb00c02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b57d5bbc1c307fb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:20.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.391+0000 7fb01a59c640 1 -- 172.21.15.69:0/207955948 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00c037070 con 0x7fb01c161710 2024-04-25T10:59:20.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.391+0000 7fb01a59c640 1 -- 172.21.15.69:0/207955948 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb00c032440 con 0x7fb01c161710 2024-04-25T10:59:20.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.391+0000 7fb01a59c640 1 -- 172.21.15.69:0/207955948 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00c0327f0 con 0x7fb01c161710 2024-04-25T10:59:20.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.391+0000 7fb0210e7640 1 -- 172.21.15.69:0/207955948 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb01c161710 msgr2=0x7fb01c15e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:20.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.391+0000 7fb0210e7640 1 --2- 172.21.15.69:0/207955948 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb01c161710 0x7fb01c15e080 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0097a0 tx=0x7fb00c02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:59:20.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.391+0000 7fb0210e7640 1 -- 172.21.15.69:0/207955948 shutdown_connections 2024-04-25T10:59:20.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.391+0000 7fb0210e7640 1 --2- 172.21.15.69:0/207955948 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb01c161710 0x7fb01c15e080 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:20.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.391+0000 7fb0210e7640 1 -- 172.21.15.69:0/207955948 >> 172.21.15.69:0/207955948 conn(0x7fb01c0d5d90 msgr2=0x7fb01c0d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:20.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.391+0000 7fb0210e7640 1 -- 172.21.15.69:0/207955948 shutdown_connections 2024-04-25T10:59:20.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.391+0000 7fb0210e7640 1 -- 172.21.15.69:0/207955948 wait complete. 2024-04-25T10:59:20.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.393+0000 7fb0210e7640 1 Processor -- start 2024-04-25T10:59:20.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.393+0000 7fb0210e7640 1 -- start start 2024-04-25T10:59:20.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.395+0000 7fb0210e7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb01c06c5c0 0x7fb01c151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:20.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.395+0000 7fb0210e7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb01c161710 0x7fb01c0cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:20.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.395+0000 7fb0210e7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb01c163fa0 0x7fb01c147c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:20.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.395+0000 7fb0210e7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c16b280 con 0x7fb01c163fa0 2024-04-25T10:59:20.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.395+0000 7fb0210e7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c16b100 con 0x7fb01c06c5c0 2024-04-25T10:59:20.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.395+0000 7fb0210e7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb01c16b400 con 0x7fb01c161710 2024-04-25T10:59:20.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.395+0000 7fb01b59e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb01c163fa0 0x7fb01c147c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:20.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.395+0000 7fb01b59e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb01c163fa0 0x7fb01c147c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41118/0 (socket says 172.21.15.69:41118) 2024-04-25T10:59:20.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.395+0000 7fb01b59e640 1 -- 172.21.15.69:0/3699653254 learned_addr learned my addr 172.21.15.69:0/3699653254 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:20.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.395+0000 7fb01ad9d640 1 --2- 172.21.15.69:0/3699653254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb01c06c5c0 0x7fb01c151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:20.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.395+0000 7fb013fff640 1 --2- 172.21.15.69:0/3699653254 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb01c161710 0x7fb01c0cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:20.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.395+0000 7fb01b59e640 1 -- 172.21.15.69:0/3699653254 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb01c161710 msgr2=0x7fb01c0cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:20.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.395+0000 7fb01b59e640 1 --2- 172.21.15.69:0/3699653254 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb01c161710 0x7fb01c0cd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:20.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.395+0000 7fb01b59e640 1 -- 172.21.15.69:0/3699653254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb01c06c5c0 msgr2=0x7fb01c151290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:20.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.395+0000 7fb01b59e640 1 --2- 172.21.15.69:0/3699653254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb01c06c5c0 0x7fb01c151290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:20.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.395+0000 7fb01b59e640 1 -- 172.21.15.69:0/3699653254 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01c0c9a70 con 0x7fb01c163fa0 2024-04-25T10:59:20.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.395+0000 7fb01b59e640 1 --2- 172.21.15.69:0/3699653254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb01c163fa0 0x7fb01c147c20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb01400cfb0 tx=0x7fb01400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:20.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.395+0000 7fb0137fe640 1 -- 172.21.15.69:0/3699653254 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014017070 con 0x7fb01c163fa0 2024-04-25T10:59:20.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.395+0000 7fb0137fe640 1 -- 172.21.15.69:0/3699653254 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb014012400 con 0x7fb01c163fa0 2024-04-25T10:59:20.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.395+0000 7fb0210e7640 1 -- 172.21.15.69:0/3699653254 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb01c1507b0 con 0x7fb01c163fa0 2024-04-25T10:59:20.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.395+0000 7fb0137fe640 1 -- 172.21.15.69:0/3699653254 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0140127f0 con 0x7fb01c163fa0 2024-04-25T10:59:20.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.395+0000 7fb0210e7640 1 -- 172.21.15.69:0/3699653254 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb01c150d70 con 0x7fb01c163fa0 2024-04-25T10:59:20.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.397+0000 7fb0210e7640 1 -- 172.21.15.69:0/3699653254 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb01c050740 con 0x7fb01c163fa0 2024-04-25T10:59:20.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.397+0000 7fb0137fe640 1 -- 172.21.15.69:0/3699653254 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb01401a070 con 0x7fb01c163fa0 2024-04-25T10:59:20.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.397+0000 7fb0137fe640 1 --2- 172.21.15.69:0/3699653254 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7faffc0744b0 0x7faffc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:20.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.397+0000 7fb0137fe640 1 -- 172.21.15.69:0/3699653254 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fb014095030 con 0x7fb01c163fa0 2024-04-25T10:59:20.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.397+0000 7fb01ad9d640 1 --2- 172.21.15.69:0/3699653254 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7faffc0744b0 0x7faffc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:20.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.398+0000 7fb01ad9d640 1 --2- 172.21.15.69:0/3699653254 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7faffc0744b0 0x7faffc0769c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb00c034000 tx=0x7fb00c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:20.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.402+0000 7fb0137fe640 1 -- 172.21.15.69:0/3699653254 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb01c1609b0 con 0x7fb01c163fa0 2024-04-25T10:59:20.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.612+0000 7fb0210e7640 1 -- 172.21.15.69:0/3699653254 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb01c0cfbe0 con 0x7fb01c163fa0 2024-04-25T10:59:20.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.633+0000 7fb0137fe640 1 -- 172.21.15.69:0/3699653254 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) ==== 118+0+0 (secure 0 0 0) 0x7fb01405f180 con 0x7fb01c163fa0 2024-04-25T10:59:20.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.635+0000 7fb0210e7640 1 -- 172.21.15.69:0/3699653254 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7faffc0744b0 msgr2=0x7faffc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:20.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.635+0000 7fb0210e7640 1 --2- 172.21.15.69:0/3699653254 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7faffc0744b0 0x7faffc0769c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb00c034000 tx=0x7fb00c02e000 comp rx=0 tx=0).stop 2024-04-25T10:59:20.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.635+0000 7fb0210e7640 1 -- 172.21.15.69:0/3699653254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb01c163fa0 msgr2=0x7fb01c147c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:20.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.635+0000 7fb0210e7640 1 --2- 172.21.15.69:0/3699653254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb01c163fa0 0x7fb01c147c20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb01400cfb0 tx=0x7fb01400a430 comp rx=0 tx=0).stop 2024-04-25T10:59:20.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.635+0000 7fb0210e7640 1 -- 172.21.15.69:0/3699653254 shutdown_connections 2024-04-25T10:59:20.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.635+0000 7fb0210e7640 1 --2- 172.21.15.69:0/3699653254 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7faffc0744b0 0x7faffc0769c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:20.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.635+0000 7fb0210e7640 1 --2- 172.21.15.69:0/3699653254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb01c163fa0 0x7fb01c147c20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:20.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.635+0000 7fb0210e7640 1 --2- 172.21.15.69:0/3699653254 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb01c161710 0x7fb01c0cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:20.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.635+0000 7fb0210e7640 1 --2- 172.21.15.69:0/3699653254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb01c06c5c0 0x7fb01c151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:20.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.635+0000 7fb0210e7640 1 -- 172.21.15.69:0/3699653254 >> 172.21.15.69:0/3699653254 conn(0x7fb01c0d5d90 msgr2=0x7fb01c15e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:20.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.636+0000 7fb0210e7640 1 -- 172.21.15.69:0/3699653254 shutdown_connections 2024-04-25T10:59:20.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.636+0000 7fb0210e7640 1 -- 172.21.15.69:0/3699653254 wait complete. 2024-04-25T10:59:20.647 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:59:20.647 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:59:20.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.755+0000 7fad8eeee640 1 Processor -- start 2024-04-25T10:59:20.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.755+0000 7fad8eeee640 1 -- start start 2024-04-25T10:59:20.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.757+0000 7fad8eeee640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fad880b2e60 0x7fad880d3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:20.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.757+0000 7fad8eeee640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fad88164020 0x7fad88164430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:20.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.757+0000 7fad8eeee640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad88164970 con 0x7fad880d1e20 2024-04-25T10:59:20.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.757+0000 7fad8eeee640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fad880d23f0 con 0x7fad880b2e60 2024-04-25T10:59:20.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.757+0000 7fad8eeee640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fad880d26d0 con 0x7fad88164020 2024-04-25T10:59:20.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.757+0000 7fad87fff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fad880b2e60 0x7fad880d3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:20.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.757+0000 7fad87fff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fad880b2e60 0x7fad880d3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41120/0 (socket says 172.21.15.69:41120) 2024-04-25T10:59:20.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.757+0000 7fad8cc8a640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fad880d1e20 0x7fad8808d770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:57130/0 (socket says 172.21.15.69:57130) 2024-04-25T10:59:20.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.757+0000 7fad87fff640 1 -- 172.21.15.69:0/2367964508 learned_addr learned my addr 172.21.15.69:0/2367964508 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:20.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.757+0000 7fad8d48b640 1 --2- 172.21.15.69:0/2367964508 >> v2:172.21.15.146:3300/0 conn(0x7fad88164020 0x7fad88164430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:20.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.757+0000 7fad87fff640 1 -- 172.21.15.69:0/2367964508 >> v2:172.21.15.146:3300/0 conn(0x7fad88164020 msgr2=0x7fad88164430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:20.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.757+0000 7fad87fff640 1 --2- 172.21.15.69:0/2367964508 >> v2:172.21.15.146:3300/0 conn(0x7fad88164020 0x7fad88164430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:20.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.757+0000 7fad87fff640 1 -- 172.21.15.69:0/2367964508 >> v1:172.21.15.69:6789/0 conn(0x7fad880d1e20 legacy=0x7fad8808d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:20.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.757+0000 7fad87fff640 1 -- 172.21.15.69:0/2367964508 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad880d29b0 con 0x7fad880b2e60 2024-04-25T10:59:20.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.757+0000 7fad877fe640 1 -- 172.21.15.69:0/2367964508 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3583359859 0 0) 0x7fad78001110 con 0x7fad880d1e20 2024-04-25T10:59:20.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.758+0000 7fad87fff640 1 --2- 172.21.15.69:0/2367964508 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad880b2e60 0x7fad880d3490 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fad8806cfd0 tx=0x7fad7c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38841bacf05a4447 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:20.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.758+0000 7fad877fe640 1 -- 172.21.15.69:0/2367964508 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c037070 con 0x7fad880b2e60 2024-04-25T10:59:20.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.758+0000 7fad877fe640 1 -- 172.21.15.69:0/2367964508 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad7c02ae50 con 0x7fad880b2e60 2024-04-25T10:59:20.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.758+0000 7fad877fe640 1 -- 172.21.15.69:0/2367964508 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c032860 con 0x7fad880b2e60 2024-04-25T10:59:20.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.759+0000 7fad8eeee640 1 -- 172.21.15.69:0/2367964508 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad880b2e60 msgr2=0x7fad880d3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:20.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.759+0000 7fad8eeee640 1 --2- 172.21.15.69:0/2367964508 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad880b2e60 0x7fad880d3490 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fad8806cfd0 tx=0x7fad7c02a300 comp rx=0 tx=0).stop 2024-04-25T10:59:20.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.759+0000 7fad8eeee640 1 -- 172.21.15.69:0/2367964508 shutdown_connections 2024-04-25T10:59:20.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.759+0000 7fad8eeee640 1 --2- 172.21.15.69:0/2367964508 >> v2:172.21.15.146:3300/0 conn(0x7fad88164020 0x7fad88164430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:20.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.759+0000 7fad8eeee640 1 --2- 172.21.15.69:0/2367964508 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad880b2e60 0x7fad880d3490 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:20.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.759+0000 7fad8eeee640 1 -- 172.21.15.69:0/2367964508 >> 172.21.15.69:0/2367964508 conn(0x7fad880d0530 msgr2=0x7fad880d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:20.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.759+0000 7fad8eeee640 1 -- 172.21.15.69:0/2367964508 shutdown_connections 2024-04-25T10:59:20.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.759+0000 7fad8eeee640 1 -- 172.21.15.69:0/2367964508 wait complete. 2024-04-25T10:59:20.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.760+0000 7fad8eeee640 1 Processor -- start 2024-04-25T10:59:20.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.761+0000 7fad8eeee640 1 -- start start 2024-04-25T10:59:20.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.762+0000 7fad8eeee640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad880b2e60 0x7fad881f5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:20.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.762+0000 7fad8eeee640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad880d1e20 0x7fad880b8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:20.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.762+0000 7fad8eeee640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fad88164020 0x7fad880b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:20.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.762+0000 7fad8eeee640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fad880ce070 con 0x7fad880d1e20 2024-04-25T10:59:20.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.762+0000 7fad8eeee640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fad880d2f70 con 0x7fad880b2e60 2024-04-25T10:59:20.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.762+0000 7fad8eeee640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fad880ce1f0 con 0x7fad88164020 2024-04-25T10:59:20.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.762+0000 7fad87fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad880d1e20 0x7fad880b8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:20.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.762+0000 7fad87fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad880d1e20 0x7fad880b8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41122/0 (socket says 172.21.15.69:41122) 2024-04-25T10:59:20.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.762+0000 7fad87fff640 1 -- 172.21.15.69:0/1031031387 learned_addr learned my addr 172.21.15.69:0/1031031387 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:20.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.762+0000 7fad8d48b640 1 --2- 172.21.15.69:0/1031031387 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fad88164020 0x7fad880b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:20.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.762+0000 7fad8cc8a640 1 --2- 172.21.15.69:0/1031031387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad880b2e60 0x7fad881f5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:20.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.762+0000 7fad87fff640 1 -- 172.21.15.69:0/1031031387 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fad88164020 msgr2=0x7fad880b0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:20.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.762+0000 7fad87fff640 1 --2- 172.21.15.69:0/1031031387 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fad88164020 0x7fad880b0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:20.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.762+0000 7fad87fff640 1 -- 172.21.15.69:0/1031031387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad880b2e60 msgr2=0x7fad881f5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:20.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.762+0000 7fad87fff640 1 --2- 172.21.15.69:0/1031031387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad880b2e60 0x7fad881f5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:20.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.762+0000 7fad87fff640 1 -- 172.21.15.69:0/1031031387 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad880b4670 con 0x7fad880d1e20 2024-04-25T10:59:20.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.763+0000 7fad87fff640 1 --2- 172.21.15.69:0/1031031387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad880d1e20 0x7fad880b8000 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fad7c035040 tx=0x7fad7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:20.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.763+0000 7fad857fa640 1 -- 172.21.15.69:0/1031031387 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c037040 con 0x7fad880d1e20 2024-04-25T10:59:20.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.763+0000 7fad857fa640 1 -- 172.21.15.69:0/1031031387 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad7c004030 con 0x7fad880d1e20 2024-04-25T10:59:20.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.763+0000 7fad8eeee640 1 -- 172.21.15.69:0/1031031387 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad880b4950 con 0x7fad880d1e20 2024-04-25T10:59:20.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.763+0000 7fad857fa640 1 -- 172.21.15.69:0/1031031387 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c004420 con 0x7fad880d1e20 2024-04-25T10:59:20.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.763+0000 7fad8eeee640 1 -- 172.21.15.69:0/1031031387 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad88158720 con 0x7fad880d1e20 2024-04-25T10:59:20.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.764+0000 7fad8eeee640 1 -- 172.21.15.69:0/1031031387 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad88050740 con 0x7fad880d1e20 2024-04-25T10:59:20.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.764+0000 7fad857fa640 1 -- 172.21.15.69:0/1031031387 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fad7c03a070 con 0x7fad880d1e20 2024-04-25T10:59:20.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.765+0000 7fad857fa640 1 --2- 172.21.15.69:0/1031031387 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fad740744e0 0x7fad740769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:20.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.765+0000 7fad8cc8a640 1 --2- 172.21.15.69:0/1031031387 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fad740744e0 0x7fad740769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:20.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.765+0000 7fad857fa640 1 -- 172.21.15.69:0/1031031387 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fad7c0b58a0 con 0x7fad880d1e20 2024-04-25T10:59:20.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.765+0000 7fad8cc8a640 1 --2- 172.21.15.69:0/1031031387 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fad740744e0 0x7fad740769f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fad78000f60 tx=0x7fad78009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:20.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.770+0000 7fad857fa640 1 -- 172.21.15.69:0/1031031387 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fad7c041020 con 0x7fad880d1e20 2024-04-25T10:59:20.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:20.981+0000 7fad8eeee640 1 -- 172.21.15.69:0/1031031387 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fad88071000 con 0x7fad880d1e20 2024-04-25T10:59:21.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.635+0000 7fad857fa640 1 -- 172.21.15.69:0/1031031387 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) ==== 122+0+0 (secure 0 0 0) 0x7fad7c0800c0 con 0x7fad880d1e20 2024-04-25T10:59:21.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.637+0000 7fad8eeee640 1 -- 172.21.15.69:0/1031031387 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fad740744e0 msgr2=0x7fad740769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:21.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.637+0000 7fad8eeee640 1 --2- 172.21.15.69:0/1031031387 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fad740744e0 0x7fad740769f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fad78000f60 tx=0x7fad78009040 comp rx=0 tx=0).stop 2024-04-25T10:59:21.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.637+0000 7fad8eeee640 1 -- 172.21.15.69:0/1031031387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad880d1e20 msgr2=0x7fad880b8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:21.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.637+0000 7fad8eeee640 1 --2- 172.21.15.69:0/1031031387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad880d1e20 0x7fad880b8000 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fad7c035040 tx=0x7fad7c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:59:21.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.637+0000 7fad8eeee640 1 -- 172.21.15.69:0/1031031387 shutdown_connections 2024-04-25T10:59:21.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.637+0000 7fad8eeee640 1 --2- 172.21.15.69:0/1031031387 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fad740744e0 0x7fad740769f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:21.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.637+0000 7fad8eeee640 1 --2- 172.21.15.69:0/1031031387 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fad88164020 0x7fad880b0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:21.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.637+0000 7fad8eeee640 1 --2- 172.21.15.69:0/1031031387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad880d1e20 0x7fad880b8000 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:21.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.637+0000 7fad8eeee640 1 --2- 172.21.15.69:0/1031031387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad880b2e60 0x7fad881f5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:21.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.637+0000 7fad8eeee640 1 -- 172.21.15.69:0/1031031387 >> 172.21.15.69:0/1031031387 conn(0x7fad880d0530 msgr2=0x7fad88158f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:21.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.637+0000 7fad8eeee640 1 -- 172.21.15.69:0/1031031387 shutdown_connections 2024-04-25T10:59:21.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.637+0000 7fad8eeee640 1 -- 172.21.15.69:0/1031031387 wait complete. 2024-04-25T10:59:21.650 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:59:21.650 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:59:21.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.758+0000 7f46d33b2640 1 Processor -- start 2024-04-25T10:59:21.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.758+0000 7f46d33b2640 1 -- start start 2024-04-25T10:59:21.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.759+0000 7f46d33b2640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f46cc0b8290 0x7f46cc160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:21.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.759+0000 7f46d33b2640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f46cc15df80 0x7f46cc160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:21.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.759+0000 7f46d33b2640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46cc1615f0 con 0x7f46cc164b00 2024-04-25T10:59:21.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.759+0000 7f46d33b2640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f46cc164670 con 0x7f46cc15df80 2024-04-25T10:59:21.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.759+0000 7f46d33b2640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f46cc164020 con 0x7f46cc0b8290 2024-04-25T10:59:21.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.759+0000 7f46d194f640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f46cc15df80 0x7f46cc160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:21.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.760+0000 7f46d194f640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f46cc15df80 0x7f46cc160dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41130/0 (socket says 172.21.15.69:41130) 2024-04-25T10:59:21.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.760+0000 7f46d194f640 1 -- 172.21.15.69:0/1493562709 learned_addr learned my addr 172.21.15.69:0/1493562709 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:21.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.760+0000 7f46d094d640 1 --2- 172.21.15.69:0/1493562709 >> v2:172.21.15.146:3300/0 conn(0x7f46cc0b8290 0x7f46cc160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:21.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.760+0000 7f46d194f640 1 -- 172.21.15.69:0/1493562709 >> v2:172.21.15.146:3300/0 conn(0x7f46cc0b8290 msgr2=0x7f46cc160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:21.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.760+0000 7f46d194f640 1 --2- 172.21.15.69:0/1493562709 >> v2:172.21.15.146:3300/0 conn(0x7f46cc0b8290 0x7f46cc160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:21.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.760+0000 7f46d194f640 1 -- 172.21.15.69:0/1493562709 >> v1:172.21.15.146:6789/0 conn(0x7f46cc164b00 legacy=0x7f46cc0b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:21.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.760+0000 7f46d194f640 1 -- 172.21.15.69:0/1493562709 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46cc164300 con 0x7f46cc15df80 2024-04-25T10:59:21.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.760+0000 7f46d194f640 1 --2- 172.21.15.69:0/1493562709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46cc15df80 0x7f46cc160dd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f46cc0d2540 tx=0x7f46c402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1229f09eb640fd99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:21.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.761+0000 7f46c3fff640 1 -- 172.21.15.69:0/1493562709 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c4036070 con 0x7f46cc15df80 2024-04-25T10:59:21.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.761+0000 7f46c3fff640 1 -- 172.21.15.69:0/1493562709 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46c4031600 con 0x7f46cc15df80 2024-04-25T10:59:21.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.761+0000 7f46c3fff640 1 -- 172.21.15.69:0/1493562709 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c40319d0 con 0x7f46cc15df80 2024-04-25T10:59:21.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.761+0000 7f46d33b2640 1 -- 172.21.15.69:0/1493562709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46cc15df80 msgr2=0x7f46cc160dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:21.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.761+0000 7f46d33b2640 1 --2- 172.21.15.69:0/1493562709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46cc15df80 0x7f46cc160dd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f46cc0d2540 tx=0x7f46c402a730 comp rx=0 tx=0).stop 2024-04-25T10:59:21.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.761+0000 7f46d33b2640 1 -- 172.21.15.69:0/1493562709 shutdown_connections 2024-04-25T10:59:21.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.761+0000 7f46d33b2640 1 --2- 172.21.15.69:0/1493562709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46cc15df80 0x7f46cc160dd0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:21.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.761+0000 7f46d33b2640 1 --2- 172.21.15.69:0/1493562709 >> v2:172.21.15.146:3300/0 conn(0x7f46cc0b8290 0x7f46cc160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:21.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.761+0000 7f46d33b2640 1 -- 172.21.15.69:0/1493562709 >> 172.21.15.69:0/1493562709 conn(0x7f46cc0d2bc0 msgr2=0x7f46cc0d2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:21.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.761+0000 7f46d33b2640 1 -- 172.21.15.69:0/1493562709 shutdown_connections 2024-04-25T10:59:21.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.761+0000 7f46d33b2640 1 -- 172.21.15.69:0/1493562709 wait complete. 2024-04-25T10:59:21.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.763+0000 7f46d33b2640 1 Processor -- start 2024-04-25T10:59:21.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.763+0000 7f46d33b2640 1 -- start start 2024-04-25T10:59:21.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.764+0000 7f46d33b2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46cc15df80 0x7f46cc0c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:21.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.764+0000 7f46d33b2640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f46cc164b00 0x7f46cc15a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:21.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.764+0000 7f46d33b2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46cc15a640 0x7f46cc154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:21.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.764+0000 7f46d33b2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc0b4a00 con 0x7f46cc15df80 2024-04-25T10:59:21.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.764+0000 7f46d33b2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc0b4880 con 0x7f46cc15a640 2024-04-25T10:59:21.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.764+0000 7f46d33b2640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f46cc0b4b80 con 0x7f46cc164b00 2024-04-25T10:59:21.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.764+0000 7f46d114e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46cc15df80 0x7f46cc0c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:21.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.765+0000 7f46d114e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46cc15df80 0x7f46cc0c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41146/0 (socket says 172.21.15.69:41146) 2024-04-25T10:59:21.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.765+0000 7f46d114e640 1 -- 172.21.15.69:0/1424199139 learned_addr learned my addr 172.21.15.69:0/1424199139 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:21.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.765+0000 7f46d114e640 1 -- 172.21.15.69:0/1424199139 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f46cc164b00 msgr2=0x7f46cc15a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:21.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.765+0000 7f46d194f640 1 --2- 172.21.15.69:0/1424199139 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46cc15a640 0x7f46cc154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:21.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.765+0000 7f46d114e640 1 --2- 172.21.15.69:0/1424199139 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f46cc164b00 0x7f46cc15a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:21.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.765+0000 7f46d114e640 1 -- 172.21.15.69:0/1424199139 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46cc15a640 msgr2=0x7f46cc154c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:21.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.765+0000 7f46d114e640 1 --2- 172.21.15.69:0/1424199139 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46cc15a640 0x7f46cc154c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:21.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.765+0000 7f46d114e640 1 -- 172.21.15.69:0/1424199139 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46cc156d00 con 0x7f46cc15df80 2024-04-25T10:59:21.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.765+0000 7f46d114e640 1 --2- 172.21.15.69:0/1424199139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46cc15df80 0x7f46cc0c8cc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f46c800c3f0 tx=0x7f46c800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:21.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.765+0000 7f46c1ffb640 1 -- 172.21.15.69:0/1424199139 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c80052d0 con 0x7f46cc15df80 2024-04-25T10:59:21.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.765+0000 7f46c1ffb640 1 -- 172.21.15.69:0/1424199139 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46c8005490 con 0x7f46cc15df80 2024-04-25T10:59:21.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.765+0000 7f46d33b2640 1 -- 172.21.15.69:0/1424199139 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46cc1548a0 con 0x7f46cc15df80 2024-04-25T10:59:21.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.765+0000 7f46c1ffb640 1 -- 172.21.15.69:0/1424199139 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8007260 con 0x7f46cc15df80 2024-04-25T10:59:21.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.765+0000 7f46d33b2640 1 -- 172.21.15.69:0/1424199139 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46cc152d40 con 0x7f46cc15df80 2024-04-25T10:59:21.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.766+0000 7f46d33b2640 1 -- 172.21.15.69:0/1424199139 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46cc050740 con 0x7f46cc15df80 2024-04-25T10:59:21.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.767+0000 7f46c1ffb640 1 -- 172.21.15.69:0/1424199139 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f46c8007400 con 0x7f46cc15df80 2024-04-25T10:59:21.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.767+0000 7f46c1ffb640 1 --2- 172.21.15.69:0/1424199139 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f46b00744b0 0x7f46b00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:21.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.767+0000 7f46d094d640 1 --2- 172.21.15.69:0/1424199139 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f46b00744b0 0x7f46b00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:21.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.768+0000 7f46c1ffb640 1 -- 172.21.15.69:0/1424199139 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f46c80946e0 con 0x7f46cc15df80 2024-04-25T10:59:21.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.768+0000 7f46d094d640 1 --2- 172.21.15.69:0/1424199139 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f46b00744b0 0x7f46b00769c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f46bc006fd0 tx=0x7f46bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:21.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.772+0000 7f46c1ffb640 1 -- 172.21.15.69:0/1424199139 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f46c805ee70 con 0x7f46cc15df80 2024-04-25T10:59:21.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.984+0000 7f46d33b2640 1 -- 172.21.15.69:0/1424199139 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46cc0d4170 con 0x7f46cc15df80 2024-04-25T10:59:21.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.985+0000 7f46c1ffb640 1 -- 172.21.15.69:0/1424199139 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5178 (secure 0 0 0) 0x7f46c805e770 con 0x7f46cc15df80 2024-04-25T10:59:21.986 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:21.986 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/2361834023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2361834023},{"type":"v1","addr":"172.21.15.146:6833","nonce":2361834023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1848276963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1848276963},{"type":"v1","addr":"172.21.15.69:6837","nonce":1848276963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1826742846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1826742846},{"type":"v1","addr":"172.21.15.69:6836","nonce":1826742846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:59:18.976232+0000","modified":"2024-04-25T10:59:20.983284+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6835/908008797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":908008797},{"type":"v1","addr":"172.21.15.146:6835","nonce":908008797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-04-25T10:59:21.986 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 6 2024-04-25T10:59:21.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.987+0000 7f46d33b2640 1 -- 172.21.15.69:0/1424199139 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f46b00744b0 msgr2=0x7f46b00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:21.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.987+0000 7f46d33b2640 1 --2- 172.21.15.69:0/1424199139 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f46b00744b0 0x7f46b00769c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f46bc006fd0 tx=0x7f46bc008040 comp rx=0 tx=0).stop 2024-04-25T10:59:21.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.987+0000 7f46d33b2640 1 -- 172.21.15.69:0/1424199139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46cc15df80 msgr2=0x7f46cc0c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:21.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.987+0000 7f46d33b2640 1 --2- 172.21.15.69:0/1424199139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46cc15df80 0x7f46cc0c8cc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f46c800c3f0 tx=0x7f46c800c8b0 comp rx=0 tx=0).stop 2024-04-25T10:59:21.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.987+0000 7f46d33b2640 1 -- 172.21.15.69:0/1424199139 shutdown_connections 2024-04-25T10:59:21.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.987+0000 7f46d33b2640 1 --2- 172.21.15.69:0/1424199139 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f46b00744b0 0x7f46b00769c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:21.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.987+0000 7f46d33b2640 1 --2- 172.21.15.69:0/1424199139 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46cc15a640 0x7f46cc154c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:21.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.987+0000 7f46d33b2640 1 --2- 172.21.15.69:0/1424199139 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f46cc164b00 0x7f46cc15a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:21.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.987+0000 7f46d33b2640 1 --2- 172.21.15.69:0/1424199139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46cc15df80 0x7f46cc0c8cc0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:21.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.987+0000 7f46d33b2640 1 -- 172.21.15.69:0/1424199139 >> 172.21.15.69:0/1424199139 conn(0x7f46cc0d2bc0 msgr2=0x7f46cc0d2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:21.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.987+0000 7f46d33b2640 1 -- 172.21.15.69:0/1424199139 shutdown_connections 2024-04-25T10:59:21.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:21.987+0000 7f46d33b2640 1 -- 172.21.15.69:0/1424199139 wait complete. 2024-04-25T10:59:21.999 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:59:22.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.105+0000 7fe59a227640 1 Processor -- start 2024-04-25T10:59:22.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.105+0000 7fe59a227640 1 -- start start 2024-04-25T10:59:22.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.106+0000 7fe59a227640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe5940d0e00 0x7fe5940d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:22.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.106+0000 7fe59a227640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe59406c5c0 0x7fe59406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:22.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.106+0000 7fe59a227640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe59406cf10 con 0x7fe5940b7810 2024-04-25T10:59:22.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.106+0000 7fe59a227640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fe5940d5a60 con 0x7fe5940d0e00 2024-04-25T10:59:22.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.106+0000 7fe59a227640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe5940d1850 con 0x7fe59406c5c0 2024-04-25T10:59:22.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.106+0000 7fe592ffd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe59406c5c0 0x7fe59406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:22.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.106+0000 7fe5937fe640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe5940d0e00 0x7fe5940d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:22.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.106+0000 7fe592ffd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe59406c5c0 0x7fe59406c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.69:58032/0 (socket says 172.21.15.69:58032) 2024-04-25T10:59:22.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.106+0000 7fe592ffd640 1 -- 172.21.15.69:0/1095466903 learned_addr learned my addr 172.21.15.69:0/1095466903 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:22.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.107+0000 7fe592ffd640 1 -- 172.21.15.69:0/1095466903 >> v2:172.21.15.146:3300/0 conn(0x7fe5940d0e00 msgr2=0x7fe5940d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:22.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.107+0000 7fe592ffd640 1 --2- 172.21.15.69:0/1095466903 >> v2:172.21.15.146:3300/0 conn(0x7fe5940d0e00 0x7fe5940d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:22.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.107+0000 7fe592ffd640 1 -- 172.21.15.69:0/1095466903 >> v1:172.21.15.146:6789/0 conn(0x7fe5940b7810 legacy=0x7fe5940b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:22.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.107+0000 7fe592ffd640 1 -- 172.21.15.69:0/1095466903 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe594160de0 con 0x7fe59406c5c0 2024-04-25T10:59:22.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.107+0000 7fe592ffd640 1 --2- 172.21.15.69:0/1095466903 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe59406c5c0 0x7fe59406c9d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe588009870 tx=0x7fe58802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=259893ccf1bd4a1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:22.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.108+0000 7fe5927fc640 1 -- 172.21.15.69:0/1095466903 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe588037070 con 0x7fe59406c5c0 2024-04-25T10:59:22.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.108+0000 7fe5927fc640 1 -- 172.21.15.69:0/1095466903 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe588032440 con 0x7fe59406c5c0 2024-04-25T10:59:22.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.108+0000 7fe5927fc640 1 -- 172.21.15.69:0/1095466903 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe588032810 con 0x7fe59406c5c0 2024-04-25T10:59:22.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.108+0000 7fe59a227640 1 -- 172.21.15.69:0/1095466903 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe59406c5c0 msgr2=0x7fe59406c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:22.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.108+0000 7fe59a227640 1 --2- 172.21.15.69:0/1095466903 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe59406c5c0 0x7fe59406c9d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe588009870 tx=0x7fe58802a270 comp rx=0 tx=0).stop 2024-04-25T10:59:22.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.108+0000 7fe59a227640 1 -- 172.21.15.69:0/1095466903 shutdown_connections 2024-04-25T10:59:22.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.108+0000 7fe59a227640 1 --2- 172.21.15.69:0/1095466903 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe59406c5c0 0x7fe59406c9d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:22.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.108+0000 7fe59a227640 1 --2- 172.21.15.69:0/1095466903 >> v2:172.21.15.146:3300/0 conn(0x7fe5940d0e00 0x7fe5940d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:22.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.108+0000 7fe59a227640 1 -- 172.21.15.69:0/1095466903 >> 172.21.15.69:0/1095466903 conn(0x7fe5940d5d90 msgr2=0x7fe5940d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:22.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.108+0000 7fe59a227640 1 -- 172.21.15.69:0/1095466903 shutdown_connections 2024-04-25T10:59:22.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.108+0000 7fe59a227640 1 -- 172.21.15.69:0/1095466903 wait complete. 2024-04-25T10:59:22.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.110+0000 7fe59a227640 1 Processor -- start 2024-04-25T10:59:22.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.110+0000 7fe59a227640 1 -- start start 2024-04-25T10:59:22.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.111+0000 7fe59a227640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe59406c5c0 0x7fe594164270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:22.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.111+0000 7fe59a227640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5940b7810 0x7fe59414f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:22.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.111+0000 7fe59a227640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe5940d0e00 0x7fe5941ebb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:22.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.111+0000 7fe59a227640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe594160ad0 con 0x7fe5940b7810 2024-04-25T10:59:22.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.111+0000 7fe59a227640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe5941613a0 con 0x7fe59406c5c0 2024-04-25T10:59:22.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.111+0000 7fe59a227640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe5940b5320 con 0x7fe5940d0e00 2024-04-25T10:59:22.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.112+0000 7fe592ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5940b7810 0x7fe59414f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:22.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.112+0000 7fe592ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5940b7810 0x7fe59414f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41150/0 (socket says 172.21.15.69:41150) 2024-04-25T10:59:22.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.112+0000 7fe592ffd640 1 -- 172.21.15.69:0/2947294659 learned_addr learned my addr 172.21.15.69:0/2947294659 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:22.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.112+0000 7fe5937fe640 1 --2- 172.21.15.69:0/2947294659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe59406c5c0 0x7fe594164270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:22.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.112+0000 7fe593fff640 1 --2- 172.21.15.69:0/2947294659 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe5940d0e00 0x7fe5941ebb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:22.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.112+0000 7fe592ffd640 1 -- 172.21.15.69:0/2947294659 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe5940d0e00 msgr2=0x7fe5941ebb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:22.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.112+0000 7fe592ffd640 1 --2- 172.21.15.69:0/2947294659 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe5940d0e00 0x7fe5941ebb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:22.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.112+0000 7fe592ffd640 1 -- 172.21.15.69:0/2947294659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe59406c5c0 msgr2=0x7fe594164270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:22.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.112+0000 7fe592ffd640 1 --2- 172.21.15.69:0/2947294659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe59406c5c0 0x7fe594164270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:22.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.112+0000 7fe592ffd640 1 -- 172.21.15.69:0/2947294659 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe594162f80 con 0x7fe5940b7810 2024-04-25T10:59:22.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.112+0000 7fe592ffd640 1 --2- 172.21.15.69:0/2947294659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5940b7810 0x7fe59414f1c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe588000c00 tx=0x7fe5880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:22.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.112+0000 7fe57bfff640 1 -- 172.21.15.69:0/2947294659 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe588037040 con 0x7fe5940b7810 2024-04-25T10:59:22.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.112+0000 7fe57bfff640 1 -- 172.21.15.69:0/2947294659 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe588042070 con 0x7fe5940b7810 2024-04-25T10:59:22.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.112+0000 7fe57bfff640 1 -- 172.21.15.69:0/2947294659 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe588030880 con 0x7fe5940b7810 2024-04-25T10:59:22.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.112+0000 7fe59a227640 1 -- 172.21.15.69:0/2947294659 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe594163260 con 0x7fe5940b7810 2024-04-25T10:59:22.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.112+0000 7fe59a227640 1 -- 172.21.15.69:0/2947294659 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe594163820 con 0x7fe5940b7810 2024-04-25T10:59:22.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.113+0000 7fe59a227640 1 -- 172.21.15.69:0/2947294659 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe594050740 con 0x7fe5940b7810 2024-04-25T10:59:22.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.114+0000 7fe57bfff640 1 -- 172.21.15.69:0/2947294659 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe588030a20 con 0x7fe5940b7810 2024-04-25T10:59:22.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.114+0000 7fe57bfff640 1 --2- 172.21.15.69:0/2947294659 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe5800744b0 0x7fe5800769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:22.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.114+0000 7fe57bfff640 1 -- 172.21.15.69:0/2947294659 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe5880b5a00 con 0x7fe5940b7810 2024-04-25T10:59:22.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.114+0000 7fe5937fe640 1 --2- 172.21.15.69:0/2947294659 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe5800744b0 0x7fe5800769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:22.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.115+0000 7fe5937fe640 1 --2- 172.21.15.69:0/2947294659 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe5800744b0 0x7fe5800769c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe584009aa0 tx=0x7fe584009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:22.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.119+0000 7fe57bfff640 1 -- 172.21.15.69:0/2947294659 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe58807fb40 con 0x7fe5940b7810 2024-04-25T10:59:22.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.294+0000 7fe59a227640 1 -- 172.21.15.69:0/2947294659 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe594162d10 con 0x7fe5940b7810 2024-04-25T10:59:22.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.295+0000 7fe57bfff640 1 -- 172.21.15.69:0/2947294659 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7fe588088210 con 0x7fe5940b7810 2024-04-25T10:59:22.296 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:22.296 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":20,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T10:59:20.630151+0000","last_up_change":"2024-04-25T10:59:05.546839+0000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:59:06.422674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:59:16.727980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:59:17.938486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6825","nonce":1876101065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6827","nonce":1876101065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6831","nonce":1876101065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6829","nonce":1876101065}]},"public_addr":"172.21.15.69:6825/1876101065","cluster_addr":"172.21.15.69:6827/1876101065","heartbeat_back_addr":"172.21.15.69:6831/1876101065","heartbeat_front_addr":"172.21.15.69:6829/1876101065","state":["exists","up"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6801","nonce":1806438563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6803","nonce":1806438563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6807","nonce":1806438563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6805","nonce":1806438563}]},"public_addr":"172.21.15.69:6801/1806438563","cluster_addr":"172.21.15.69:6803/1806438563","heartbeat_back_addr":"172.21.15.69:6807/1806438563","heartbeat_front_addr":"172.21.15.69:6805/1806438563","state":["exists","up"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6817","nonce":1593617983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6819","nonce":1593617983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6823","nonce":1593617983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6821","nonce":1593617983}]},"public_addr":"172.21.15.69:6817/1593617983","cluster_addr":"172.21.15.69:6819/1593617983","heartbeat_back_addr":"172.21.15.69:6823/1593617983","heartbeat_front_addr":"172.21.15.69:6821/1593617983","state":["exists","up"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6809","nonce":3950661294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6811","nonce":3950661294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6815","nonce":3950661294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6813","nonce":3950661294}]},"public_addr":"172.21.15.69:6809/3950661294","cluster_addr":"172.21.15.69:6811/3950661294","heartbeat_back_addr":"172.21.15.69:6815/3950661294","heartbeat_front_addr":"172.21.15.69:6813/3950661294","state":["exists","up"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6801","nonce":2508929872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6803","nonce":2508929872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6807","nonce":2508929872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6805","nonce":2508929872}]},"public_addr":"172.21.15.146:6801/2508929872","cluster_addr":"172.21.15.146:6803/2508929872","heartbeat_back_addr":"172.21.15.146:6807/2508929872","heartbeat_front_addr":"172.21.15.146:6805/2508929872","state":["exists","up"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6811","nonce":1310911138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6816","nonce":1310911138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6823","nonce":1310911138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6821","nonce":1310911138}]},"public_addr":"172.21.15.146:6811/1310911138","cluster_addr":"172.21.15.146:6816/1310911138","heartbeat_back_addr":"172.21.15.146:6823/1310911138","heartbeat_front_addr":"172.21.15.146:6821/1310911138","state":["exists","up"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6810","nonce":634688021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6813","nonce":634688021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6820","nonce":634688021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6817","nonce":634688021}]},"public_addr":"172.21.15.146:6810/634688021","cluster_addr":"172.21.15.146:6813/634688021","heartbeat_back_addr":"172.21.15.146:6820/634688021","heartbeat_front_addr":"172.21.15.146:6817/634688021","state":["exists","up"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6825","nonce":3814215935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6827","nonce":3814215935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6831","nonce":3814215935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6829","nonce":3814215935}]},"public_addr":"172.21.15.146:6825/3814215935","cluster_addr":"172.21.15.146:6827/3814215935","heartbeat_back_addr":"172.21.15.146:6831/3814215935","heartbeat_front_addr":"172.21.15.146:6829/3814215935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.820200+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.850189+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.764560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.866248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.807897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.869881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.782310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.827131+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:59:22.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.297+0000 7fe59a227640 1 -- 172.21.15.69:0/2947294659 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe5800744b0 msgr2=0x7fe5800769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:22.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.297+0000 7fe59a227640 1 --2- 172.21.15.69:0/2947294659 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe5800744b0 0x7fe5800769c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe584009aa0 tx=0x7fe584009190 comp rx=0 tx=0).stop 2024-04-25T10:59:22.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.297+0000 7fe59a227640 1 -- 172.21.15.69:0/2947294659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5940b7810 msgr2=0x7fe59414f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:22.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.297+0000 7fe59a227640 1 --2- 172.21.15.69:0/2947294659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5940b7810 0x7fe59414f1c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe588000c00 tx=0x7fe5880046c0 comp rx=0 tx=0).stop 2024-04-25T10:59:22.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.297+0000 7fe59a227640 1 -- 172.21.15.69:0/2947294659 shutdown_connections 2024-04-25T10:59:22.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.297+0000 7fe59a227640 1 --2- 172.21.15.69:0/2947294659 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe5800744b0 0x7fe5800769c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:22.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.297+0000 7fe59a227640 1 --2- 172.21.15.69:0/2947294659 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe5940d0e00 0x7fe5941ebb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:22.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.297+0000 7fe59a227640 1 --2- 172.21.15.69:0/2947294659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5940b7810 0x7fe59414f1c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:22.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.297+0000 7fe59a227640 1 --2- 172.21.15.69:0/2947294659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe59406c5c0 0x7fe594164270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:22.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.297+0000 7fe59a227640 1 -- 172.21.15.69:0/2947294659 >> 172.21.15.69:0/2947294659 conn(0x7fe5940d5d90 msgr2=0x7fe5940b6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:22.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.298+0000 7fe59a227640 1 -- 172.21.15.69:0/2947294659 shutdown_connections 2024-04-25T10:59:22.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.298+0000 7fe59a227640 1 -- 172.21.15.69:0/2947294659 wait complete. 2024-04-25T10:59:22.309 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:59:22.310 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:59:22.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.417+0000 7f375e042640 1 Processor -- start 2024-04-25T10:59:22.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.417+0000 7f375e042640 1 -- start start 2024-04-25T10:59:22.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.418+0000 7f375e042640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3758160700 0x7f37580b1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:22.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.418+0000 7f375e042640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f37580b17c0 0x7f375806c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:22.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.418+0000 7f375e042640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f375806cb00 0x7f37580ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:22.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.418+0000 7f375e042640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f37580b7a90 con 0x7f37580b17c0 2024-04-25T10:59:22.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.418+0000 7f375e042640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f37580b1bd0 con 0x7f375806cb00 2024-04-25T10:59:22.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.418+0000 7f375e042640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f375806cf10 con 0x7f3758160700 2024-04-25T10:59:22.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.418+0000 7f3756ffd640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f37580b17c0 0x7f375806c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:22.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.419+0000 7f3756ffd640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f37580b17c0 0x7f375806c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41162/0 (socket says 172.21.15.69:41162) 2024-04-25T10:59:22.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.419+0000 7f3756ffd640 1 -- 172.21.15.69:0/3463250617 learned_addr learned my addr 172.21.15.69:0/3463250617 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:22.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.419+0000 7f3757fff640 1 --2- 172.21.15.69:0/3463250617 >> v2:172.21.15.146:3300/0 conn(0x7f375806cb00 0x7f37580ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:22.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.419+0000 7f37577fe640 1 --2- 172.21.15.69:0/3463250617 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3758160700 0x7f37580b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:22.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.419+0000 7f3756ffd640 1 -- 172.21.15.69:0/3463250617 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3758160700 msgr2=0x7f37580b1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:22.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.419+0000 7f3756ffd640 1 --2- 172.21.15.69:0/3463250617 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3758160700 0x7f37580b1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:22.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.419+0000 7f3756ffd640 1 -- 172.21.15.69:0/3463250617 >> v2:172.21.15.146:3300/0 conn(0x7f375806cb00 msgr2=0x7f37580ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:22.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.419+0000 7f3756ffd640 1 --2- 172.21.15.69:0/3463250617 >> v2:172.21.15.146:3300/0 conn(0x7f375806cb00 0x7f37580ceb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:22.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.419+0000 7f3756ffd640 1 -- 172.21.15.69:0/3463250617 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f375806d1a0 con 0x7f37580b17c0 2024-04-25T10:59:22.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.419+0000 7f3756ffd640 1 --2- 172.21.15.69:0/3463250617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37580b17c0 0x7f375806c5c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f37580c5c70 tx=0x7f374c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304290c30352562b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:22.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.420+0000 7f37567fc640 1 -- 172.21.15.69:0/3463250617 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c037070 con 0x7f37580b17c0 2024-04-25T10:59:22.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.420+0000 7f37567fc640 1 -- 172.21.15.69:0/3463250617 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f374c032440 con 0x7f37580b17c0 2024-04-25T10:59:22.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.420+0000 7f37567fc640 1 -- 172.21.15.69:0/3463250617 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c032810 con 0x7f37580b17c0 2024-04-25T10:59:22.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.420+0000 7f375e042640 1 -- 172.21.15.69:0/3463250617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37580b17c0 msgr2=0x7f375806c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:22.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.420+0000 7f375e042640 1 --2- 172.21.15.69:0/3463250617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37580b17c0 0x7f375806c5c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f37580c5c70 tx=0x7f374c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:59:22.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.420+0000 7f375e042640 1 -- 172.21.15.69:0/3463250617 shutdown_connections 2024-04-25T10:59:22.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.420+0000 7f375e042640 1 --2- 172.21.15.69:0/3463250617 >> v2:172.21.15.146:3300/0 conn(0x7f375806cb00 0x7f37580ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:22.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.420+0000 7f375e042640 1 --2- 172.21.15.69:0/3463250617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37580b17c0 0x7f375806c5c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:22.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.420+0000 7f375e042640 1 --2- 172.21.15.69:0/3463250617 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3758160700 0x7f37580b1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:22.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.420+0000 7f375e042640 1 -- 172.21.15.69:0/3463250617 >> 172.21.15.69:0/3463250617 conn(0x7f375809b7d0 msgr2=0x7f37580b7050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:22.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.420+0000 7f375e042640 1 -- 172.21.15.69:0/3463250617 shutdown_connections 2024-04-25T10:59:22.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.420+0000 7f375e042640 1 -- 172.21.15.69:0/3463250617 wait complete. 2024-04-25T10:59:22.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.422+0000 7f375e042640 1 Processor -- start 2024-04-25T10:59:22.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.422+0000 7f375e042640 1 -- start start 2024-04-25T10:59:22.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.424+0000 7f375e042640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f375806cb00 0x7f3758155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:22.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.424+0000 7f375e042640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37580b17c0 0x7f37580cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:22.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.424+0000 7f375e042640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3758160700 0x7f37580ce300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:22.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.424+0000 7f375e042640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f375806da40 con 0x7f375806cb00 2024-04-25T10:59:22.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.424+0000 7f375e042640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f375806d760 con 0x7f37580b17c0 2024-04-25T10:59:22.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.424+0000 7f375e042640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f375806dd20 con 0x7f3758160700 2024-04-25T10:59:22.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.424+0000 7f37577fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f375806cb00 0x7f3758155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:22.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.424+0000 7f37577fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f375806cb00 0x7f3758155990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41170/0 (socket says 172.21.15.69:41170) 2024-04-25T10:59:22.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.424+0000 7f37577fe640 1 -- 172.21.15.69:0/2219674860 learned_addr learned my addr 172.21.15.69:0/2219674860 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:22.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.424+0000 7f3756ffd640 1 --2- 172.21.15.69:0/2219674860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37580b17c0 0x7f37580cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:22.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.424+0000 7f3757fff640 1 --2- 172.21.15.69:0/2219674860 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3758160700 0x7f37580ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:22.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.424+0000 7f37577fe640 1 -- 172.21.15.69:0/2219674860 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3758160700 msgr2=0x7f37580ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:22.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.424+0000 7f37577fe640 1 --2- 172.21.15.69:0/2219674860 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3758160700 0x7f37580ce300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:22.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.424+0000 7f37577fe640 1 -- 172.21.15.69:0/2219674860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37580b17c0 msgr2=0x7f37580cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:22.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.424+0000 7f37577fe640 1 --2- 172.21.15.69:0/2219674860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37580b17c0 0x7f37580cddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:22.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.424+0000 7f37577fe640 1 -- 172.21.15.69:0/2219674860 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3758159a80 con 0x7f375806cb00 2024-04-25T10:59:22.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.424+0000 7f37577fe640 1 --2- 172.21.15.69:0/2219674860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f375806cb00 0x7f3758155990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f374800d600 tx=0x7f374800dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:22.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.424+0000 7f373ffff640 1 -- 172.21.15.69:0/2219674860 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3748004240 con 0x7f375806cb00 2024-04-25T10:59:22.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.424+0000 7f373ffff640 1 -- 172.21.15.69:0/2219674860 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37480043e0 con 0x7f375806cb00 2024-04-25T10:59:22.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.424+0000 7f375e042640 1 -- 172.21.15.69:0/2219674860 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3758159f80 con 0x7f375806cb00 2024-04-25T10:59:22.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.424+0000 7f373ffff640 1 -- 172.21.15.69:0/2219674860 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3748003830 con 0x7f375806cb00 2024-04-25T10:59:22.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.424+0000 7f375e042640 1 -- 172.21.15.69:0/2219674860 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f375815a540 con 0x7f375806cb00 2024-04-25T10:59:22.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.426+0000 7f375e042640 1 -- 172.21.15.69:0/2219674860 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3758050740 con 0x7f375806cb00 2024-04-25T10:59:22.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.426+0000 7f373ffff640 1 -- 172.21.15.69:0/2219674860 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f37480039d0 con 0x7f375806cb00 2024-04-25T10:59:22.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.426+0000 7f373ffff640 1 --2- 172.21.15.69:0/2219674860 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f37440744b0 0x7f37440769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:22.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.427+0000 7f3756ffd640 1 --2- 172.21.15.69:0/2219674860 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f37440744b0 0x7f37440769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:22.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.427+0000 7f373ffff640 1 -- 172.21.15.69:0/2219674860 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3748095330 con 0x7f375806cb00 2024-04-25T10:59:22.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.427+0000 7f3756ffd640 1 --2- 172.21.15.69:0/2219674860 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f37440744b0 0x7f37440769c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f374c034000 tx=0x7f374c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:22.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.431+0000 7f373ffff640 1 -- 172.21.15.69:0/2219674860 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f374805f500 con 0x7f375806cb00 2024-04-25T10:59:22.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.602+0000 7f375e042640 1 -- 172.21.15.69:0/2219674860 --> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f375815ef50 con 0x7f37440744b0 2024-04-25T10:59:22.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.608+0000 7f373ffff640 1 -- 172.21.15.69:0/2219674860 <== mgr.4107 v2:172.21.15.69:6832/1107421267 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251936 (secure 0 0 0) 0x7f375815ef50 con 0x7f37440744b0 2024-04-25T10:59:22.612 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:22.613 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-04-25T10:59:22.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.611+0000 7f375e042640 1 -- 172.21.15.69:0/2219674860 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f37440744b0 msgr2=0x7f37440769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:22.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.611+0000 7f375e042640 1 --2- 172.21.15.69:0/2219674860 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f37440744b0 0x7f37440769c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f374c034000 tx=0x7f374c02e000 comp rx=0 tx=0).stop 2024-04-25T10:59:22.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.611+0000 7f375e042640 1 -- 172.21.15.69:0/2219674860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f375806cb00 msgr2=0x7f3758155990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:22.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.611+0000 7f375e042640 1 --2- 172.21.15.69:0/2219674860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f375806cb00 0x7f3758155990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f374800d600 tx=0x7f374800dac0 comp rx=0 tx=0).stop 2024-04-25T10:59:22.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.611+0000 7f375e042640 1 -- 172.21.15.69:0/2219674860 shutdown_connections 2024-04-25T10:59:22.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.611+0000 7f375e042640 1 --2- 172.21.15.69:0/2219674860 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f37440744b0 0x7f37440769c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:22.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.611+0000 7f375e042640 1 --2- 172.21.15.69:0/2219674860 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3758160700 0x7f37580ce300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:22.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.611+0000 7f375e042640 1 --2- 172.21.15.69:0/2219674860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37580b17c0 0x7f37580cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:22.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.611+0000 7f375e042640 1 --2- 172.21.15.69:0/2219674860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f375806cb00 0x7f3758155990 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:22.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.611+0000 7f375e042640 1 -- 172.21.15.69:0/2219674860 >> 172.21.15.69:0/2219674860 conn(0x7f375809b7d0 msgr2=0x7f3758070760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:22.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.611+0000 7f375e042640 1 -- 172.21.15.69:0/2219674860 shutdown_connections 2024-04-25T10:59:22.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.611+0000 7f375e042640 1 -- 172.21.15.69:0/2219674860 wait complete. 2024-04-25T10:59:22.623 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-25T10:59:22.406342+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":180,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218948,"kb_used_data":2764,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755772,"statfs":{"total":773094113280,"available":772869910528,"internally_reserved":0,"allocated":2830336,"data_stored":1721528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.794625"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228976+0000","last_change":"2024-04-25T10:59:19.640957+0000","last_active":"2024-04-25T10:59:21.228976+0000","last_peered":"2024-04-25T10:59:21.228976+0000","last_clean":"2024-04-25T10:59:21.228976+0000","last_became_active":"2024-04-25T10:59:19.640610+0000","last_became_peered":"2024-04-25T10:59:19.640610+0000","last_unstale":"2024-04-25T10:59:21.228976+0000","last_undegraded":"2024-04-25T10:59:21.228976+0000","last_fullsized":"2024-04-25T10:59:21.228976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:42.867755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636604+0000","last_change":"2024-04-25T10:59:19.644473+0000","last_active":"2024-04-25T10:59:20.636604+0000","last_peered":"2024-04-25T10:59:20.636604+0000","last_clean":"2024-04-25T10:59:20.636604+0000","last_became_active":"2024-04-25T10:59:19.644151+0000","last_became_peered":"2024-04-25T10:59:19.644151+0000","last_unstale":"2024-04-25T10:59:20.636604+0000","last_undegraded":"2024-04-25T10:59:20.636604+0000","last_fullsized":"2024-04-25T10:59:20.636604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:07.522883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227761+0000","last_change":"2024-04-25T10:59:19.630984+0000","last_active":"2024-04-25T10:59:21.227761+0000","last_peered":"2024-04-25T10:59:21.227761+0000","last_clean":"2024-04-25T10:59:21.227761+0000","last_became_active":"2024-04-25T10:59:19.630699+0000","last_became_peered":"2024-04-25T10:59:19.630699+0000","last_unstale":"2024-04-25T10:59:21.227761+0000","last_undegraded":"2024-04-25T10:59:21.227761+0000","last_fullsized":"2024-04-25T10:59:21.227761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:59.213210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228437+0000","last_change":"2024-04-25T10:59:19.649708+0000","last_active":"2024-04-25T10:59:21.228437+0000","last_peered":"2024-04-25T10:59:21.228437+0000","last_clean":"2024-04-25T10:59:21.228437+0000","last_became_active":"2024-04-25T10:59:19.649262+0000","last_became_peered":"2024-04-25T10:59:19.649262+0000","last_unstale":"2024-04-25T10:59:21.228437+0000","last_undegraded":"2024-04-25T10:59:21.228437+0000","last_fullsized":"2024-04-25T10:59:21.228437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:52.334437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228519+0000","last_change":"2024-04-25T10:59:19.642490+0000","last_active":"2024-04-25T10:59:21.228519+0000","last_peered":"2024-04-25T10:59:21.228519+0000","last_clean":"2024-04-25T10:59:21.228519+0000","last_became_active":"2024-04-25T10:59:19.642122+0000","last_became_peered":"2024-04-25T10:59:19.642122+0000","last_unstale":"2024-04-25T10:59:21.228519+0000","last_undegraded":"2024-04-25T10:59:21.228519+0000","last_fullsized":"2024-04-25T10:59:21.228519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:14.189727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228892+0000","last_change":"2024-04-25T10:59:19.651061+0000","last_active":"2024-04-25T10:59:21.228892+0000","last_peered":"2024-04-25T10:59:21.228892+0000","last_clean":"2024-04-25T10:59:21.228892+0000","last_became_active":"2024-04-25T10:59:19.650710+0000","last_became_peered":"2024-04-25T10:59:19.650710+0000","last_unstale":"2024-04-25T10:59:21.228892+0000","last_undegraded":"2024-04-25T10:59:21.228892+0000","last_fullsized":"2024-04-25T10:59:21.228892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:45.328749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227817+0000","last_change":"2024-04-25T10:59:19.646862+0000","last_active":"2024-04-25T10:59:21.227817+0000","last_peered":"2024-04-25T10:59:21.227817+0000","last_clean":"2024-04-25T10:59:21.227817+0000","last_became_active":"2024-04-25T10:59:19.646553+0000","last_became_peered":"2024-04-25T10:59:19.646553+0000","last_unstale":"2024-04-25T10:59:21.227817+0000","last_undegraded":"2024-04-25T10:59:21.227817+0000","last_fullsized":"2024-04-25T10:59:21.227817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:52.334437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227508+0000","last_change":"2024-04-25T10:59:19.648086+0000","last_active":"2024-04-25T10:59:21.227508+0000","last_peered":"2024-04-25T10:59:21.227508+0000","last_clean":"2024-04-25T10:59:21.227508+0000","last_became_active":"2024-04-25T10:59:19.647702+0000","last_became_peered":"2024-04-25T10:59:19.647702+0000","last_unstale":"2024-04-25T10:59:21.227508+0000","last_undegraded":"2024-04-25T10:59:21.227508+0000","last_fullsized":"2024-04-25T10:59:21.227508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:59.213210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228983+0000","last_change":"2024-04-25T10:59:19.655202+0000","last_active":"2024-04-25T10:59:21.228983+0000","last_peered":"2024-04-25T10:59:21.228983+0000","last_clean":"2024-04-25T10:59:21.228983+0000","last_became_active":"2024-04-25T10:59:19.654871+0000","last_became_peered":"2024-04-25T10:59:19.654871+0000","last_unstale":"2024-04-25T10:59:21.228983+0000","last_undegraded":"2024-04-25T10:59:21.228983+0000","last_fullsized":"2024-04-25T10:59:21.228983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:50.893634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638094+0000","last_change":"2024-04-25T10:59:19.651928+0000","last_active":"2024-04-25T10:59:20.638094+0000","last_peered":"2024-04-25T10:59:20.638094+0000","last_clean":"2024-04-25T10:59:20.638094+0000","last_became_active":"2024-04-25T10:59:19.651618+0000","last_became_peered":"2024-04-25T10:59:19.651618+0000","last_unstale":"2024-04-25T10:59:20.638094+0000","last_undegraded":"2024-04-25T10:59:20.638094+0000","last_fullsized":"2024-04-25T10:59:20.638094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:32:26.553467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638187+0000","last_change":"2024-04-25T10:59:19.652810+0000","last_active":"2024-04-25T10:59:20.638187+0000","last_peered":"2024-04-25T10:59:20.638187+0000","last_clean":"2024-04-25T10:59:20.638187+0000","last_became_active":"2024-04-25T10:59:19.652118+0000","last_became_peered":"2024-04-25T10:59:19.652118+0000","last_unstale":"2024-04-25T10:59:20.638187+0000","last_undegraded":"2024-04-25T10:59:20.638187+0000","last_fullsized":"2024-04-25T10:59:20.638187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:59.213210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638421+0000","last_change":"2024-04-25T10:59:19.645548+0000","last_active":"2024-04-25T10:59:20.638421+0000","last_peered":"2024-04-25T10:59:20.638421+0000","last_clean":"2024-04-25T10:59:20.638421+0000","last_became_active":"2024-04-25T10:59:19.645231+0000","last_became_peered":"2024-04-25T10:59:19.645231+0000","last_unstale":"2024-04-25T10:59:20.638421+0000","last_undegraded":"2024-04-25T10:59:20.638421+0000","last_fullsized":"2024-04-25T10:59:20.638421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:59.512598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227723+0000","last_change":"2024-04-25T10:59:19.640736+0000","last_active":"2024-04-25T10:59:21.227723+0000","last_peered":"2024-04-25T10:59:21.227723+0000","last_clean":"2024-04-25T10:59:21.227723+0000","last_became_active":"2024-04-25T10:59:19.640448+0000","last_became_peered":"2024-04-25T10:59:19.640448+0000","last_unstale":"2024-04-25T10:59:21.227723+0000","last_undegraded":"2024-04-25T10:59:21.227723+0000","last_fullsized":"2024-04-25T10:59:21.227723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:15.132862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228215+0000","last_change":"2024-04-25T10:59:19.634929+0000","last_active":"2024-04-25T10:59:21.228215+0000","last_peered":"2024-04-25T10:59:21.228215+0000","last_clean":"2024-04-25T10:59:21.228215+0000","last_became_active":"2024-04-25T10:59:19.634640+0000","last_became_peered":"2024-04-25T10:59:19.634640+0000","last_unstale":"2024-04-25T10:59:21.228215+0000","last_undegraded":"2024-04-25T10:59:21.228215+0000","last_fullsized":"2024-04-25T10:59:21.228215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:49.314956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636375+0000","last_change":"2024-04-25T10:59:19.648597+0000","last_active":"2024-04-25T10:59:20.636375+0000","last_peered":"2024-04-25T10:59:20.636375+0000","last_clean":"2024-04-25T10:59:20.636375+0000","last_became_active":"2024-04-25T10:59:19.648280+0000","last_became_peered":"2024-04-25T10:59:19.648280+0000","last_unstale":"2024-04-25T10:59:20.636375+0000","last_undegraded":"2024-04-25T10:59:20.636375+0000","last_fullsized":"2024-04-25T10:59:20.636375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:58.824966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228034+0000","last_change":"2024-04-25T10:59:19.647183+0000","last_active":"2024-04-25T10:59:21.228034+0000","last_peered":"2024-04-25T10:59:21.228034+0000","last_clean":"2024-04-25T10:59:21.228034+0000","last_became_active":"2024-04-25T10:59:19.646563+0000","last_became_peered":"2024-04-25T10:59:19.646563+0000","last_unstale":"2024-04-25T10:59:21.228034+0000","last_undegraded":"2024-04-25T10:59:21.228034+0000","last_fullsized":"2024-04-25T10:59:21.228034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:57.344787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228532+0000","last_change":"2024-04-25T10:59:19.649207+0000","last_active":"2024-04-25T10:59:21.228532+0000","last_peered":"2024-04-25T10:59:21.228532+0000","last_clean":"2024-04-25T10:59:21.228532+0000","last_became_active":"2024-04-25T10:59:19.647280+0000","last_became_peered":"2024-04-25T10:59:19.647280+0000","last_unstale":"2024-04-25T10:59:21.228532+0000","last_undegraded":"2024-04-25T10:59:21.228532+0000","last_fullsized":"2024-04-25T10:59:21.228532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:32.457404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.185793+0000","last_change":"2024-04-25T10:59:19.640810+0000","last_active":"2024-04-25T10:59:21.185793+0000","last_peered":"2024-04-25T10:59:21.185793+0000","last_clean":"2024-04-25T10:59:21.185793+0000","last_became_active":"2024-04-25T10:59:19.640524+0000","last_became_peered":"2024-04-25T10:59:19.640524+0000","last_unstale":"2024-04-25T10:59:21.185793+0000","last_undegraded":"2024-04-25T10:59:21.185793+0000","last_fullsized":"2024-04-25T10:59:21.185793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:58.824966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228703+0000","last_change":"2024-04-25T10:59:19.641698+0000","last_active":"2024-04-25T10:59:21.228703+0000","last_peered":"2024-04-25T10:59:21.228703+0000","last_clean":"2024-04-25T10:59:21.228703+0000","last_became_active":"2024-04-25T10:59:19.640996+0000","last_became_peered":"2024-04-25T10:59:19.640996+0000","last_unstale":"2024-04-25T10:59:21.228703+0000","last_undegraded":"2024-04-25T10:59:21.228703+0000","last_fullsized":"2024-04-25T10:59:21.228703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:33.154184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.639226+0000","last_change":"2024-04-25T10:59:19.654176+0000","last_active":"2024-04-25T10:59:20.639226+0000","last_peered":"2024-04-25T10:59:20.639226+0000","last_clean":"2024-04-25T10:59:20.639226+0000","last_became_active":"2024-04-25T10:59:19.653175+0000","last_became_peered":"2024-04-25T10:59:19.653175+0000","last_unstale":"2024-04-25T10:59:20.639226+0000","last_undegraded":"2024-04-25T10:59:20.639226+0000","last_fullsized":"2024-04-25T10:59:20.639226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:33.050282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227935+0000","last_change":"2024-04-25T10:59:19.642959+0000","last_active":"2024-04-25T10:59:21.227935+0000","last_peered":"2024-04-25T10:59:21.227935+0000","last_clean":"2024-04-25T10:59:21.227935+0000","last_became_active":"2024-04-25T10:59:19.642671+0000","last_became_peered":"2024-04-25T10:59:19.642671+0000","last_unstale":"2024-04-25T10:59:21.227935+0000","last_undegraded":"2024-04-25T10:59:21.227935+0000","last_fullsized":"2024-04-25T10:59:21.227935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:15.132862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228503+0000","last_change":"2024-04-25T10:59:19.658039+0000","last_active":"2024-04-25T10:59:21.228503+0000","last_peered":"2024-04-25T10:59:21.228503+0000","last_clean":"2024-04-25T10:59:21.228503+0000","last_became_active":"2024-04-25T10:59:19.657194+0000","last_became_peered":"2024-04-25T10:59:19.657194+0000","last_unstale":"2024-04-25T10:59:21.228503+0000","last_undegraded":"2024-04-25T10:59:21.228503+0000","last_fullsized":"2024-04-25T10:59:21.228503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:19.485287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227782+0000","last_change":"2024-04-25T10:59:19.645451+0000","last_active":"2024-04-25T10:59:21.227782+0000","last_peered":"2024-04-25T10:59:21.227782+0000","last_clean":"2024-04-25T10:59:21.227782+0000","last_became_active":"2024-04-25T10:59:19.645121+0000","last_became_peered":"2024-04-25T10:59:19.645121+0000","last_unstale":"2024-04-25T10:59:21.227782+0000","last_undegraded":"2024-04-25T10:59:21.227782+0000","last_fullsized":"2024-04-25T10:59:21.227782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:07.522883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228328+0000","last_change":"2024-04-25T10:59:19.646916+0000","last_active":"2024-04-25T10:59:21.228328+0000","last_peered":"2024-04-25T10:59:21.228328+0000","last_clean":"2024-04-25T10:59:21.228328+0000","last_became_active":"2024-04-25T10:59:19.646228+0000","last_became_peered":"2024-04-25T10:59:19.646228+0000","last_unstale":"2024-04-25T10:59:21.228328+0000","last_undegraded":"2024-04-25T10:59:21.228328+0000","last_fullsized":"2024-04-25T10:59:21.228328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:17.431757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228328+0000","last_change":"2024-04-25T10:59:19.644795+0000","last_active":"2024-04-25T10:59:21.228328+0000","last_peered":"2024-04-25T10:59:21.228328+0000","last_clean":"2024-04-25T10:59:21.228328+0000","last_became_active":"2024-04-25T10:59:19.644503+0000","last_became_peered":"2024-04-25T10:59:19.644503+0000","last_unstale":"2024-04-25T10:59:21.228328+0000","last_undegraded":"2024-04-25T10:59:21.228328+0000","last_fullsized":"2024-04-25T10:59:21.228328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:17.431757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638291+0000","last_change":"2024-04-25T10:59:19.650630+0000","last_active":"2024-04-25T10:59:20.638291+0000","last_peered":"2024-04-25T10:59:20.638291+0000","last_clean":"2024-04-25T10:59:20.638291+0000","last_became_active":"2024-04-25T10:59:19.650312+0000","last_became_peered":"2024-04-25T10:59:19.650312+0000","last_unstale":"2024-04-25T10:59:20.638291+0000","last_undegraded":"2024-04-25T10:59:20.638291+0000","last_fullsized":"2024-04-25T10:59:20.638291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:58.824966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636446+0000","last_change":"2024-04-25T10:59:19.641641+0000","last_active":"2024-04-25T10:59:20.636446+0000","last_peered":"2024-04-25T10:59:20.636446+0000","last_clean":"2024-04-25T10:59:20.636446+0000","last_became_active":"2024-04-25T10:59:19.641339+0000","last_became_peered":"2024-04-25T10:59:19.641339+0000","last_unstale":"2024-04-25T10:59:20.636446+0000","last_undegraded":"2024-04-25T10:59:20.636446+0000","last_fullsized":"2024-04-25T10:59:20.636446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:52.815596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228173+0000","last_change":"2024-04-25T10:59:19.642824+0000","last_active":"2024-04-25T10:59:21.228173+0000","last_peered":"2024-04-25T10:59:21.228173+0000","last_clean":"2024-04-25T10:59:21.228173+0000","last_became_active":"2024-04-25T10:59:19.641152+0000","last_became_peered":"2024-04-25T10:59:19.641152+0000","last_unstale":"2024-04-25T10:59:21.228173+0000","last_undegraded":"2024-04-25T10:59:21.228173+0000","last_fullsized":"2024-04-25T10:59:21.228173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:49.314956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228117+0000","last_change":"2024-04-25T10:59:19.629558+0000","last_active":"2024-04-25T10:59:21.228117+0000","last_peered":"2024-04-25T10:59:21.228117+0000","last_clean":"2024-04-25T10:59:21.228117+0000","last_became_active":"2024-04-25T10:59:19.628864+0000","last_became_peered":"2024-04-25T10:59:19.628864+0000","last_unstale":"2024-04-25T10:59:21.228117+0000","last_undegraded":"2024-04-25T10:59:21.228117+0000","last_fullsized":"2024-04-25T10:59:21.228117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:07.522883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228910+0000","last_change":"2024-04-25T10:59:19.648576+0000","last_active":"2024-04-25T10:59:21.228910+0000","last_peered":"2024-04-25T10:59:21.228910+0000","last_clean":"2024-04-25T10:59:21.228910+0000","last_became_active":"2024-04-25T10:59:19.647435+0000","last_became_peered":"2024-04-25T10:59:19.647435+0000","last_unstale":"2024-04-25T10:59:21.228910+0000","last_undegraded":"2024-04-25T10:59:21.228910+0000","last_fullsized":"2024-04-25T10:59:21.228910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:42.867755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.185603+0000","last_change":"2024-04-25T10:59:19.649031+0000","last_active":"2024-04-25T10:59:21.185603+0000","last_peered":"2024-04-25T10:59:21.185603+0000","last_clean":"2024-04-25T10:59:21.185603+0000","last_became_active":"2024-04-25T10:59:19.648698+0000","last_became_peered":"2024-04-25T10:59:19.648698+0000","last_unstale":"2024-04-25T10:59:21.185603+0000","last_undegraded":"2024-04-25T10:59:21.185603+0000","last_fullsized":"2024-04-25T10:59:21.185603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:59.213210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638942+0000","last_change":"2024-04-25T10:59:19.643547+0000","last_active":"2024-04-25T10:59:20.638942+0000","last_peered":"2024-04-25T10:59:20.638942+0000","last_clean":"2024-04-25T10:59:20.638942+0000","last_became_active":"2024-04-25T10:59:19.642802+0000","last_became_peered":"2024-04-25T10:59:19.642802+0000","last_unstale":"2024-04-25T10:59:20.638942+0000","last_undegraded":"2024-04-25T10:59:20.638942+0000","last_fullsized":"2024-04-25T10:59:20.638942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:15.132862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638161+0000","last_change":"2024-04-25T10:59:19.638090+0000","last_active":"2024-04-25T10:59:20.638161+0000","last_peered":"2024-04-25T10:59:20.638161+0000","last_clean":"2024-04-25T10:59:20.638161+0000","last_became_active":"2024-04-25T10:59:19.637745+0000","last_became_peered":"2024-04-25T10:59:19.637745+0000","last_unstale":"2024-04-25T10:59:20.638161+0000","last_undegraded":"2024-04-25T10:59:20.638161+0000","last_fullsized":"2024-04-25T10:59:20.638161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:25.274583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227963+0000","last_change":"2024-04-25T10:59:18.637294+0000","last_active":"2024-04-25T10:59:21.227963+0000","last_peered":"2024-04-25T10:59:21.227963+0000","last_clean":"2024-04-25T10:59:21.227963+0000","last_became_active":"2024-04-25T10:59:18.636963+0000","last_became_peered":"2024-04-25T10:59:18.636963+0000","last_unstale":"2024-04-25T10:59:21.227963+0000","last_undegraded":"2024-04-25T10:59:21.227963+0000","last_fullsized":"2024-04-25T10:59:21.227963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:18.298759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228183+0000","last_change":"2024-04-25T10:59:19.638014+0000","last_active":"2024-04-25T10:59:21.228183+0000","last_peered":"2024-04-25T10:59:21.228183+0000","last_clean":"2024-04-25T10:59:21.228183+0000","last_became_active":"2024-04-25T10:59:19.637471+0000","last_became_peered":"2024-04-25T10:59:19.637471+0000","last_unstale":"2024-04-25T10:59:21.228183+0000","last_undegraded":"2024-04-25T10:59:21.228183+0000","last_fullsized":"2024-04-25T10:59:21.228183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:33.050282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638880+0000","last_change":"2024-04-25T10:59:18.634982+0000","last_active":"2024-04-25T10:59:20.638880+0000","last_peered":"2024-04-25T10:59:20.638880+0000","last_clean":"2024-04-25T10:59:20.638880+0000","last_became_active":"2024-04-25T10:59:18.634658+0000","last_became_peered":"2024-04-25T10:59:18.634658+0000","last_unstale":"2024-04-25T10:59:20.638880+0000","last_undegraded":"2024-04-25T10:59:20.638880+0000","last_fullsized":"2024-04-25T10:59:20.638880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:35.341389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227940+0000","last_change":"2024-04-25T10:59:18.632617+0000","last_active":"2024-04-25T10:59:21.227940+0000","last_peered":"2024-04-25T10:59:21.227940+0000","last_clean":"2024-04-25T10:59:21.227940+0000","last_became_active":"2024-04-25T10:59:18.632294+0000","last_became_peered":"2024-04-25T10:59:18.632294+0000","last_unstale":"2024-04-25T10:59:21.227940+0000","last_undegraded":"2024-04-25T10:59:21.227940+0000","last_fullsized":"2024-04-25T10:59:21.227940+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:46.279987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227575+0000","last_change":"2024-04-25T10:59:19.632333+0000","last_active":"2024-04-25T10:59:21.227575+0000","last_peered":"2024-04-25T10:59:21.227575+0000","last_clean":"2024-04-25T10:59:21.227575+0000","last_became_active":"2024-04-25T10:59:19.632042+0000","last_became_peered":"2024-04-25T10:59:19.632042+0000","last_unstale":"2024-04-25T10:59:21.227575+0000","last_undegraded":"2024-04-25T10:59:21.227575+0000","last_fullsized":"2024-04-25T10:59:21.227575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:58.824966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228184+0000","last_change":"2024-04-25T10:59:18.638103+0000","last_active":"2024-04-25T10:59:21.228184+0000","last_peered":"2024-04-25T10:59:21.228184+0000","last_clean":"2024-04-25T10:59:21.228184+0000","last_became_active":"2024-04-25T10:59:18.636931+0000","last_became_peered":"2024-04-25T10:59:18.636931+0000","last_unstale":"2024-04-25T10:59:21.228184+0000","last_undegraded":"2024-04-25T10:59:21.228184+0000","last_fullsized":"2024-04-25T10:59:21.228184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:15.754476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636374+0000","last_change":"2024-04-25T10:59:19.633809+0000","last_active":"2024-04-25T10:59:20.636374+0000","last_peered":"2024-04-25T10:59:20.636374+0000","last_clean":"2024-04-25T10:59:20.636374+0000","last_became_active":"2024-04-25T10:59:19.632787+0000","last_became_peered":"2024-04-25T10:59:19.632787+0000","last_unstale":"2024-04-25T10:59:20.636374+0000","last_undegraded":"2024-04-25T10:59:20.636374+0000","last_fullsized":"2024-04-25T10:59:20.636374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:59.213210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638419+0000","last_change":"2024-04-25T10:59:18.641813+0000","last_active":"2024-04-25T10:59:20.638419+0000","last_peered":"2024-04-25T10:59:20.638419+0000","last_clean":"2024-04-25T10:59:20.638419+0000","last_became_active":"2024-04-25T10:59:18.641127+0000","last_became_peered":"2024-04-25T10:59:18.641127+0000","last_unstale":"2024-04-25T10:59:20.638419+0000","last_undegraded":"2024-04-25T10:59:20.638419+0000","last_fullsized":"2024-04-25T10:59:20.638419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:28.594312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228699+0000","last_change":"2024-04-25T10:59:19.645823+0000","last_active":"2024-04-25T10:59:21.228699+0000","last_peered":"2024-04-25T10:59:21.228699+0000","last_clean":"2024-04-25T10:59:21.228699+0000","last_became_active":"2024-04-25T10:59:19.645500+0000","last_became_peered":"2024-04-25T10:59:19.645500+0000","last_unstale":"2024-04-25T10:59:21.228699+0000","last_undegraded":"2024-04-25T10:59:21.228699+0000","last_fullsized":"2024-04-25T10:59:21.228699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:00.311029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228172+0000","last_change":"2024-04-25T10:59:18.641002+0000","last_active":"2024-04-25T10:59:21.228172+0000","last_peered":"2024-04-25T10:59:21.228172+0000","last_clean":"2024-04-25T10:59:21.228172+0000","last_became_active":"2024-04-25T10:59:18.640589+0000","last_became_peered":"2024-04-25T10:59:18.640589+0000","last_unstale":"2024-04-25T10:59:21.228172+0000","last_undegraded":"2024-04-25T10:59:21.228172+0000","last_fullsized":"2024-04-25T10:59:21.228172+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:24.267022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227906+0000","last_change":"2024-04-25T10:59:19.652033+0000","last_active":"2024-04-25T10:59:21.227906+0000","last_peered":"2024-04-25T10:59:21.227906+0000","last_clean":"2024-04-25T10:59:21.227906+0000","last_became_active":"2024-04-25T10:59:19.651694+0000","last_became_peered":"2024-04-25T10:59:19.651694+0000","last_unstale":"2024-04-25T10:59:21.227906+0000","last_undegraded":"2024-04-25T10:59:21.227906+0000","last_fullsized":"2024-04-25T10:59:21.227906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:58.824966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636485+0000","last_change":"2024-04-25T10:59:18.638096+0000","last_active":"2024-04-25T10:59:20.636485+0000","last_peered":"2024-04-25T10:59:20.636485+0000","last_clean":"2024-04-25T10:59:20.636485+0000","last_became_active":"2024-04-25T10:59:18.637808+0000","last_became_peered":"2024-04-25T10:59:18.637808+0000","last_unstale":"2024-04-25T10:59:20.636485+0000","last_undegraded":"2024-04-25T10:59:20.636485+0000","last_fullsized":"2024-04-25T10:59:20.636485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:10.473815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638613+0000","last_change":"2024-04-25T10:59:19.638709+0000","last_active":"2024-04-25T10:59:20.638613+0000","last_peered":"2024-04-25T10:59:20.638613+0000","last_clean":"2024-04-25T10:59:20.638613+0000","last_became_active":"2024-04-25T10:59:19.638358+0000","last_became_peered":"2024-04-25T10:59:19.638358+0000","last_unstale":"2024-04-25T10:59:20.638613+0000","last_undegraded":"2024-04-25T10:59:20.638613+0000","last_fullsized":"2024-04-25T10:59:20.638613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:46.619782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.639204+0000","last_change":"2024-04-25T10:59:18.639824+0000","last_active":"2024-04-25T10:59:20.639204+0000","last_peered":"2024-04-25T10:59:20.639204+0000","last_clean":"2024-04-25T10:59:20.639204+0000","last_became_active":"2024-04-25T10:59:18.639316+0000","last_became_peered":"2024-04-25T10:59:18.639316+0000","last_unstale":"2024-04-25T10:59:20.639204+0000","last_undegraded":"2024-04-25T10:59:20.639204+0000","last_fullsized":"2024-04-25T10:59:20.639204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:15.754476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228064+0000","last_change":"2024-04-25T10:59:19.641380+0000","last_active":"2024-04-25T10:59:21.228064+0000","last_peered":"2024-04-25T10:59:21.228064+0000","last_clean":"2024-04-25T10:59:21.228064+0000","last_became_active":"2024-04-25T10:59:19.640580+0000","last_became_peered":"2024-04-25T10:59:19.640580+0000","last_unstale":"2024-04-25T10:59:21.228064+0000","last_undegraded":"2024-04-25T10:59:21.228064+0000","last_fullsized":"2024-04-25T10:59:21.228064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:19.485287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228226+0000","last_change":"2024-04-25T10:59:18.634324+0000","last_active":"2024-04-25T10:59:21.228226+0000","last_peered":"2024-04-25T10:59:21.228226+0000","last_clean":"2024-04-25T10:59:21.228226+0000","last_became_active":"2024-04-25T10:59:18.634049+0000","last_became_peered":"2024-04-25T10:59:18.634049+0000","last_unstale":"2024-04-25T10:59:21.228226+0000","last_undegraded":"2024-04-25T10:59:21.228226+0000","last_fullsized":"2024-04-25T10:59:21.228226+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:02.619679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228376+0000","last_change":"2024-04-25T10:59:19.643539+0000","last_active":"2024-04-25T10:59:21.228376+0000","last_peered":"2024-04-25T10:59:21.228376+0000","last_clean":"2024-04-25T10:59:21.228376+0000","last_became_active":"2024-04-25T10:59:19.643182+0000","last_became_peered":"2024-04-25T10:59:19.643182+0000","last_unstale":"2024-04-25T10:59:21.228376+0000","last_undegraded":"2024-04-25T10:59:21.228376+0000","last_fullsized":"2024-04-25T10:59:21.228376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:57.344787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.229036+0000","last_change":"2024-04-25T10:59:18.643936+0000","last_active":"2024-04-25T10:59:21.229036+0000","last_peered":"2024-04-25T10:59:21.229036+0000","last_clean":"2024-04-25T10:59:21.229036+0000","last_became_active":"2024-04-25T10:59:18.643576+0000","last_became_peered":"2024-04-25T10:59:18.643576+0000","last_unstale":"2024-04-25T10:59:21.229036+0000","last_undegraded":"2024-04-25T10:59:21.229036+0000","last_fullsized":"2024-04-25T10:59:21.229036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:58.505038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228718+0000","last_change":"2024-04-25T10:59:19.652528+0000","last_active":"2024-04-25T10:59:21.228718+0000","last_peered":"2024-04-25T10:59:21.228718+0000","last_clean":"2024-04-25T10:59:21.228718+0000","last_became_active":"2024-04-25T10:59:19.651277+0000","last_became_peered":"2024-04-25T10:59:19.651277+0000","last_unstale":"2024-04-25T10:59:21.228718+0000","last_undegraded":"2024-04-25T10:59:21.228718+0000","last_fullsized":"2024-04-25T10:59:21.228718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:45.328749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.637715+0000","last_change":"2024-04-25T10:59:18.630832+0000","last_active":"2024-04-25T10:59:20.637715+0000","last_peered":"2024-04-25T10:59:20.637715+0000","last_clean":"2024-04-25T10:59:20.637715+0000","last_became_active":"2024-04-25T10:59:18.630462+0000","last_became_peered":"2024-04-25T10:59:18.630462+0000","last_unstale":"2024-04-25T10:59:20.637715+0000","last_undegraded":"2024-04-25T10:59:20.637715+0000","last_fullsized":"2024-04-25T10:59:20.637715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:46.279987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.639078+0000","last_change":"2024-04-25T10:59:19.643805+0000","last_active":"2024-04-25T10:59:20.639078+0000","last_peered":"2024-04-25T10:59:20.639078+0000","last_clean":"2024-04-25T10:59:20.639078+0000","last_became_active":"2024-04-25T10:59:19.643222+0000","last_became_peered":"2024-04-25T10:59:19.643222+0000","last_unstale":"2024-04-25T10:59:20.639078+0000","last_undegraded":"2024-04-25T10:59:20.639078+0000","last_fullsized":"2024-04-25T10:59:20.639078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:07.522883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.185571+0000","last_change":"2024-04-25T10:59:18.627083+0000","last_active":"2024-04-25T10:59:21.185571+0000","last_peered":"2024-04-25T10:59:21.185571+0000","last_clean":"2024-04-25T10:59:21.185571+0000","last_became_active":"2024-04-25T10:59:18.626450+0000","last_became_peered":"2024-04-25T10:59:18.626450+0000","last_unstale":"2024-04-25T10:59:21.185571+0000","last_undegraded":"2024-04-25T10:59:21.185571+0000","last_fullsized":"2024-04-25T10:59:21.185571+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:45.612221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228298+0000","last_change":"2024-04-25T10:59:19.653272+0000","last_active":"2024-04-25T10:59:21.228298+0000","last_peered":"2024-04-25T10:59:21.228298+0000","last_clean":"2024-04-25T10:59:21.228298+0000","last_became_active":"2024-04-25T10:59:19.652954+0000","last_became_peered":"2024-04-25T10:59:19.652954+0000","last_unstale":"2024-04-25T10:59:21.228298+0000","last_undegraded":"2024-04-25T10:59:21.228298+0000","last_fullsized":"2024-04-25T10:59:21.228298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:32.457404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638454+0000","last_change":"2024-04-25T10:59:18.630333+0000","last_active":"2024-04-25T10:59:20.638454+0000","last_peered":"2024-04-25T10:59:20.638454+0000","last_clean":"2024-04-25T10:59:20.638454+0000","last_became_active":"2024-04-25T10:59:18.629185+0000","last_became_peered":"2024-04-25T10:59:18.629185+0000","last_unstale":"2024-04-25T10:59:20.638454+0000","last_undegraded":"2024-04-25T10:59:20.638454+0000","last_fullsized":"2024-04-25T10:59:20.638454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:12.306399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228990+0000","last_change":"2024-04-25T10:59:19.654703+0000","last_active":"2024-04-25T10:59:21.228990+0000","last_peered":"2024-04-25T10:59:21.228990+0000","last_clean":"2024-04-25T10:59:21.228990+0000","last_became_active":"2024-04-25T10:59:19.654381+0000","last_became_peered":"2024-04-25T10:59:19.654381+0000","last_unstale":"2024-04-25T10:59:21.228990+0000","last_undegraded":"2024-04-25T10:59:21.228990+0000","last_fullsized":"2024-04-25T10:59:21.228990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:43.481266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.185751+0000","last_change":"2024-04-25T10:59:18.639591+0000","last_active":"2024-04-25T10:59:21.185751+0000","last_peered":"2024-04-25T10:59:21.185751+0000","last_clean":"2024-04-25T10:59:21.185751+0000","last_became_active":"2024-04-25T10:59:18.639276+0000","last_became_peered":"2024-04-25T10:59:18.639276+0000","last_unstale":"2024-04-25T10:59:21.185751+0000","last_undegraded":"2024-04-25T10:59:21.185751+0000","last_fullsized":"2024-04-25T10:59:21.185751+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:05.044315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.229162+0000","last_change":"2024-04-25T10:59:19.650496+0000","last_active":"2024-04-25T10:59:21.229162+0000","last_peered":"2024-04-25T10:59:21.229162+0000","last_clean":"2024-04-25T10:59:21.229162+0000","last_became_active":"2024-04-25T10:59:19.649725+0000","last_became_peered":"2024-04-25T10:59:19.649725+0000","last_unstale":"2024-04-25T10:59:21.229162+0000","last_undegraded":"2024-04-25T10:59:21.229162+0000","last_fullsized":"2024-04-25T10:59:21.229162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:53.860108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636578+0000","last_change":"2024-04-25T10:59:18.635647+0000","last_active":"2024-04-25T10:59:20.636578+0000","last_peered":"2024-04-25T10:59:20.636578+0000","last_clean":"2024-04-25T10:59:20.636578+0000","last_became_active":"2024-04-25T10:59:18.635264+0000","last_became_peered":"2024-04-25T10:59:18.635264+0000","last_unstale":"2024-04-25T10:59:20.636578+0000","last_undegraded":"2024-04-25T10:59:20.636578+0000","last_fullsized":"2024-04-25T10:59:20.636578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:45.612221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228718+0000","last_change":"2024-04-25T10:59:19.650869+0000","last_active":"2024-04-25T10:59:21.228718+0000","last_peered":"2024-04-25T10:59:21.228718+0000","last_clean":"2024-04-25T10:59:21.228718+0000","last_became_active":"2024-04-25T10:59:19.650541+0000","last_became_peered":"2024-04-25T10:59:19.650541+0000","last_unstale":"2024-04-25T10:59:21.228718+0000","last_undegraded":"2024-04-25T10:59:21.228718+0000","last_fullsized":"2024-04-25T10:59:21.228718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:14.189727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.185690+0000","last_change":"2024-04-25T10:59:18.637436+0000","last_active":"2024-04-25T10:59:21.185690+0000","last_peered":"2024-04-25T10:59:21.185690+0000","last_clean":"2024-04-25T10:59:21.185690+0000","last_became_active":"2024-04-25T10:59:18.637152+0000","last_became_peered":"2024-04-25T10:59:18.637152+0000","last_unstale":"2024-04-25T10:59:21.185690+0000","last_undegraded":"2024-04-25T10:59:21.185690+0000","last_fullsized":"2024-04-25T10:59:21.185690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:10.473815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228358+0000","last_change":"2024-04-25T10:59:19.655023+0000","last_active":"2024-04-25T10:59:21.228358+0000","last_peered":"2024-04-25T10:59:21.228358+0000","last_clean":"2024-04-25T10:59:21.228358+0000","last_became_active":"2024-04-25T10:59:19.654647+0000","last_became_peered":"2024-04-25T10:59:19.654647+0000","last_unstale":"2024-04-25T10:59:21.228358+0000","last_undegraded":"2024-04-25T10:59:21.228358+0000","last_fullsized":"2024-04-25T10:59:21.228358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:00.311029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":110,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227788+0000","last_change":"2024-04-25T10:59:07.839466+0000","last_active":"2024-04-25T10:59:21.227788+0000","last_peered":"2024-04-25T10:59:21.227788+0000","last_clean":"2024-04-25T10:59:21.227788+0000","last_became_active":"2024-04-25T10:59:07.836269+0000","last_became_peered":"2024-04-25T10:59:07.836269+0000","last_unstale":"2024-04-25T10:59:21.227788+0000","last_undegraded":"2024-04-25T10:59:21.227788+0000","last_fullsized":"2024-04-25T10:59:21.227788+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:06.549663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:06.549663+0000","last_clean_scrub_stamp":"2024-04-25T10:59:06.549663+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:21.575330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228265+0000","last_change":"2024-04-25T10:59:18.639570+0000","last_active":"2024-04-25T10:59:21.228265+0000","last_peered":"2024-04-25T10:59:21.228265+0000","last_clean":"2024-04-25T10:59:21.228265+0000","last_became_active":"2024-04-25T10:59:18.639231+0000","last_became_peered":"2024-04-25T10:59:18.639231+0000","last_unstale":"2024-04-25T10:59:21.228265+0000","last_undegraded":"2024-04-25T10:59:21.228265+0000","last_fullsized":"2024-04-25T10:59:21.228265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:28.594312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228305+0000","last_change":"2024-04-25T10:59:19.652991+0000","last_active":"2024-04-25T10:59:21.228305+0000","last_peered":"2024-04-25T10:59:21.228305+0000","last_clean":"2024-04-25T10:59:21.228305+0000","last_became_active":"2024-04-25T10:59:19.652680+0000","last_became_peered":"2024-04-25T10:59:19.652680+0000","last_unstale":"2024-04-25T10:59:21.228305+0000","last_undegraded":"2024-04-25T10:59:21.228305+0000","last_fullsized":"2024-04-25T10:59:21.228305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:38.297267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228139+0000","last_change":"2024-04-25T10:59:18.639571+0000","last_active":"2024-04-25T10:59:21.228139+0000","last_peered":"2024-04-25T10:59:21.228139+0000","last_clean":"2024-04-25T10:59:21.228139+0000","last_became_active":"2024-04-25T10:59:18.639065+0000","last_became_peered":"2024-04-25T10:59:18.639065+0000","last_unstale":"2024-04-25T10:59:21.228139+0000","last_undegraded":"2024-04-25T10:59:21.228139+0000","last_fullsized":"2024-04-25T10:59:21.228139+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:10.473815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227726+0000","last_change":"2024-04-25T10:59:19.644241+0000","last_active":"2024-04-25T10:59:21.227726+0000","last_peered":"2024-04-25T10:59:21.227726+0000","last_clean":"2024-04-25T10:59:21.227726+0000","last_became_active":"2024-04-25T10:59:19.643778+0000","last_became_peered":"2024-04-25T10:59:19.643778+0000","last_unstale":"2024-04-25T10:59:21.227726+0000","last_undegraded":"2024-04-25T10:59:21.227726+0000","last_fullsized":"2024-04-25T10:59:21.227726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:10.276726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.185642+0000","last_change":"2024-04-25T10:59:18.643811+0000","last_active":"2024-04-25T10:59:21.185642+0000","last_peered":"2024-04-25T10:59:21.185642+0000","last_clean":"2024-04-25T10:59:21.185642+0000","last_became_active":"2024-04-25T10:59:18.643454+0000","last_became_peered":"2024-04-25T10:59:18.643454+0000","last_unstale":"2024-04-25T10:59:21.185642+0000","last_undegraded":"2024-04-25T10:59:21.185642+0000","last_fullsized":"2024-04-25T10:59:21.185642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:09.933183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636785+0000","last_change":"2024-04-25T10:59:19.639259+0000","last_active":"2024-04-25T10:59:20.636785+0000","last_peered":"2024-04-25T10:59:20.636785+0000","last_clean":"2024-04-25T10:59:20.636785+0000","last_became_active":"2024-04-25T10:59:19.637680+0000","last_became_peered":"2024-04-25T10:59:19.637680+0000","last_unstale":"2024-04-25T10:59:20.636785+0000","last_undegraded":"2024-04-25T10:59:20.636785+0000","last_fullsized":"2024-04-25T10:59:20.636785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:33.050282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.229027+0000","last_change":"2024-04-25T10:59:18.643664+0000","last_active":"2024-04-25T10:59:21.229027+0000","last_peered":"2024-04-25T10:59:21.229027+0000","last_clean":"2024-04-25T10:59:21.229027+0000","last_became_active":"2024-04-25T10:59:18.643309+0000","last_became_peered":"2024-04-25T10:59:18.643309+0000","last_unstale":"2024-04-25T10:59:21.229027+0000","last_undegraded":"2024-04-25T10:59:21.229027+0000","last_fullsized":"2024-04-25T10:59:21.229027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:45.612221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.639084+0000","last_change":"2024-04-25T10:59:19.633507+0000","last_active":"2024-04-25T10:59:20.639084+0000","last_peered":"2024-04-25T10:59:20.639084+0000","last_clean":"2024-04-25T10:59:20.639084+0000","last_became_active":"2024-04-25T10:59:19.633182+0000","last_became_peered":"2024-04-25T10:59:19.633182+0000","last_unstale":"2024-04-25T10:59:20.639084+0000","last_undegraded":"2024-04-25T10:59:20.639084+0000","last_fullsized":"2024-04-25T10:59:20.639084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:11.481375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636672+0000","last_change":"2024-04-25T10:59:18.635772+0000","last_active":"2024-04-25T10:59:20.636672+0000","last_peered":"2024-04-25T10:59:20.636672+0000","last_clean":"2024-04-25T10:59:20.636672+0000","last_became_active":"2024-04-25T10:59:18.635486+0000","last_became_peered":"2024-04-25T10:59:18.635486+0000","last_unstale":"2024-04-25T10:59:20.636672+0000","last_undegraded":"2024-04-25T10:59:20.636672+0000","last_fullsized":"2024-04-25T10:59:20.636672+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:24.267022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228031+0000","last_change":"2024-04-25T10:59:19.648239+0000","last_active":"2024-04-25T10:59:21.228031+0000","last_peered":"2024-04-25T10:59:21.228031+0000","last_clean":"2024-04-25T10:59:21.228031+0000","last_became_active":"2024-04-25T10:59:19.646876+0000","last_became_peered":"2024-04-25T10:59:19.646876+0000","last_unstale":"2024-04-25T10:59:21.228031+0000","last_undegraded":"2024-04-25T10:59:21.228031+0000","last_fullsized":"2024-04-25T10:59:21.228031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:49.314956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228833+0000","last_change":"2024-04-25T10:59:18.643449+0000","last_active":"2024-04-25T10:59:21.228833+0000","last_peered":"2024-04-25T10:59:21.228833+0000","last_clean":"2024-04-25T10:59:21.228833+0000","last_became_active":"2024-04-25T10:59:18.643069+0000","last_became_peered":"2024-04-25T10:59:18.643069+0000","last_unstale":"2024-04-25T10:59:21.228833+0000","last_undegraded":"2024-04-25T10:59:21.228833+0000","last_fullsized":"2024-04-25T10:59:21.228833+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:57.176546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636600+0000","last_change":"2024-04-25T10:59:19.638854+0000","last_active":"2024-04-25T10:59:20.636600+0000","last_peered":"2024-04-25T10:59:20.636600+0000","last_clean":"2024-04-25T10:59:20.636600+0000","last_became_active":"2024-04-25T10:59:19.638273+0000","last_became_peered":"2024-04-25T10:59:19.638273+0000","last_unstale":"2024-04-25T10:59:20.636600+0000","last_undegraded":"2024-04-25T10:59:20.636600+0000","last_fullsized":"2024-04-25T10:59:20.636600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:15.132862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636693+0000","last_change":"2024-04-25T10:59:18.635651+0000","last_active":"2024-04-25T10:59:20.636693+0000","last_peered":"2024-04-25T10:59:20.636693+0000","last_clean":"2024-04-25T10:59:20.636693+0000","last_became_active":"2024-04-25T10:59:18.635348+0000","last_became_peered":"2024-04-25T10:59:18.635348+0000","last_unstale":"2024-04-25T10:59:20.636693+0000","last_undegraded":"2024-04-25T10:59:20.636693+0000","last_fullsized":"2024-04-25T10:59:20.636693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:58.505038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228269+0000","last_change":"2024-04-25T10:59:19.640663+0000","last_active":"2024-04-25T10:59:21.228269+0000","last_peered":"2024-04-25T10:59:21.228269+0000","last_clean":"2024-04-25T10:59:21.228269+0000","last_became_active":"2024-04-25T10:59:19.640311+0000","last_became_peered":"2024-04-25T10:59:19.640311+0000","last_unstale":"2024-04-25T10:59:21.228269+0000","last_undegraded":"2024-04-25T10:59:21.228269+0000","last_fullsized":"2024-04-25T10:59:21.228269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:57.344787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638534+0000","last_change":"2024-04-25T10:59:18.635706+0000","last_active":"2024-04-25T10:59:20.638534+0000","last_peered":"2024-04-25T10:59:20.638534+0000","last_clean":"2024-04-25T10:59:20.638534+0000","last_became_active":"2024-04-25T10:59:18.635377+0000","last_became_peered":"2024-04-25T10:59:18.635377+0000","last_unstale":"2024-04-25T10:59:20.638534+0000","last_undegraded":"2024-04-25T10:59:20.638534+0000","last_fullsized":"2024-04-25T10:59:20.638534+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:24.267022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227935+0000","last_change":"2024-04-25T10:59:19.635255+0000","last_active":"2024-04-25T10:59:21.227935+0000","last_peered":"2024-04-25T10:59:21.227935+0000","last_clean":"2024-04-25T10:59:21.227935+0000","last_became_active":"2024-04-25T10:59:19.634829+0000","last_became_peered":"2024-04-25T10:59:19.634829+0000","last_unstale":"2024-04-25T10:59:21.227935+0000","last_undegraded":"2024-04-25T10:59:21.227935+0000","last_fullsized":"2024-04-25T10:59:21.227935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:52.815596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227702+0000","last_change":"2024-04-25T10:59:18.643806+0000","last_active":"2024-04-25T10:59:21.227702+0000","last_peered":"2024-04-25T10:59:21.227702+0000","last_clean":"2024-04-25T10:59:21.227702+0000","last_became_active":"2024-04-25T10:59:18.643374+0000","last_became_peered":"2024-04-25T10:59:18.643374+0000","last_unstale":"2024-04-25T10:59:21.227702+0000","last_undegraded":"2024-04-25T10:59:21.227702+0000","last_fullsized":"2024-04-25T10:59:21.227702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:45.612221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.637981+0000","last_change":"2024-04-25T10:59:19.637663+0000","last_active":"2024-04-25T10:59:20.637981+0000","last_peered":"2024-04-25T10:59:20.637981+0000","last_clean":"2024-04-25T10:59:20.637981+0000","last_became_active":"2024-04-25T10:59:19.637309+0000","last_became_peered":"2024-04-25T10:59:19.637309+0000","last_unstale":"2024-04-25T10:59:20.637981+0000","last_undegraded":"2024-04-25T10:59:20.637981+0000","last_fullsized":"2024-04-25T10:59:20.637981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:29.601873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636484+0000","last_change":"2024-04-25T10:59:18.636340+0000","last_active":"2024-04-25T10:59:20.636484+0000","last_peered":"2024-04-25T10:59:20.636484+0000","last_clean":"2024-04-25T10:59:20.636484+0000","last_became_active":"2024-04-25T10:59:18.636036+0000","last_became_peered":"2024-04-25T10:59:18.636036+0000","last_unstale":"2024-04-25T10:59:20.636484+0000","last_undegraded":"2024-04-25T10:59:20.636484+0000","last_fullsized":"2024-04-25T10:59:20.636484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:15.754476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227666+0000","last_change":"2024-04-25T10:59:19.632935+0000","last_active":"2024-04-25T10:59:21.227666+0000","last_peered":"2024-04-25T10:59:21.227666+0000","last_clean":"2024-04-25T10:59:21.227666+0000","last_became_active":"2024-04-25T10:59:19.632658+0000","last_became_peered":"2024-04-25T10:59:19.632658+0000","last_unstale":"2024-04-25T10:59:21.227666+0000","last_undegraded":"2024-04-25T10:59:21.227666+0000","last_fullsized":"2024-04-25T10:59:21.227666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:52.815596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636560+0000","last_change":"2024-04-25T10:59:18.639304+0000","last_active":"2024-04-25T10:59:20.636560+0000","last_peered":"2024-04-25T10:59:20.636560+0000","last_clean":"2024-04-25T10:59:20.636560+0000","last_became_active":"2024-04-25T10:59:18.638939+0000","last_became_peered":"2024-04-25T10:59:18.638939+0000","last_unstale":"2024-04-25T10:59:20.636560+0000","last_undegraded":"2024-04-25T10:59:20.636560+0000","last_fullsized":"2024-04-25T10:59:20.636560+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:28.594312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227866+0000","last_change":"2024-04-25T10:59:19.631658+0000","last_active":"2024-04-25T10:59:21.227866+0000","last_peered":"2024-04-25T10:59:21.227866+0000","last_clean":"2024-04-25T10:59:21.227866+0000","last_became_active":"2024-04-25T10:59:19.631363+0000","last_became_peered":"2024-04-25T10:59:19.631363+0000","last_unstale":"2024-04-25T10:59:21.227866+0000","last_undegraded":"2024-04-25T10:59:21.227866+0000","last_fullsized":"2024-04-25T10:59:21.227866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:33.050282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228855+0000","last_change":"2024-04-25T10:59:18.638514+0000","last_active":"2024-04-25T10:59:21.228855+0000","last_peered":"2024-04-25T10:59:21.228855+0000","last_clean":"2024-04-25T10:59:21.228855+0000","last_became_active":"2024-04-25T10:59:18.638113+0000","last_became_peered":"2024-04-25T10:59:18.638113+0000","last_unstale":"2024-04-25T10:59:21.228855+0000","last_undegraded":"2024-04-25T10:59:21.228855+0000","last_fullsized":"2024-04-25T10:59:21.228855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:09.933183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638752+0000","last_change":"2024-04-25T10:59:19.652586+0000","last_active":"2024-04-25T10:59:20.638752+0000","last_peered":"2024-04-25T10:59:20.638752+0000","last_clean":"2024-04-25T10:59:20.638752+0000","last_became_active":"2024-04-25T10:59:19.652246+0000","last_became_peered":"2024-04-25T10:59:19.652246+0000","last_unstale":"2024-04-25T10:59:20.638752+0000","last_undegraded":"2024-04-25T10:59:20.638752+0000","last_fullsized":"2024-04-25T10:59:20.638752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:52.815596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228042+0000","last_change":"2024-04-25T10:59:19.649474+0000","last_active":"2024-04-25T10:59:21.228042+0000","last_peered":"2024-04-25T10:59:21.228042+0000","last_clean":"2024-04-25T10:59:21.228042+0000","last_became_active":"2024-04-25T10:59:19.649134+0000","last_became_peered":"2024-04-25T10:59:19.649134+0000","last_unstale":"2024-04-25T10:59:21.228042+0000","last_undegraded":"2024-04-25T10:59:21.228042+0000","last_fullsized":"2024-04-25T10:59:21.228042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:38.297267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228094+0000","last_change":"2024-04-25T10:59:18.641120+0000","last_active":"2024-04-25T10:59:21.228094+0000","last_peered":"2024-04-25T10:59:21.228094+0000","last_clean":"2024-04-25T10:59:21.228094+0000","last_became_active":"2024-04-25T10:59:18.640791+0000","last_became_peered":"2024-04-25T10:59:18.640791+0000","last_unstale":"2024-04-25T10:59:21.228094+0000","last_undegraded":"2024-04-25T10:59:21.228094+0000","last_fullsized":"2024-04-25T10:59:21.228094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:43.470493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228726+0000","last_change":"2024-04-25T10:59:19.654395+0000","last_active":"2024-04-25T10:59:21.228726+0000","last_peered":"2024-04-25T10:59:21.228726+0000","last_clean":"2024-04-25T10:59:21.228726+0000","last_became_active":"2024-04-25T10:59:19.654107+0000","last_became_peered":"2024-04-25T10:59:19.654107+0000","last_unstale":"2024-04-25T10:59:21.228726+0000","last_undegraded":"2024-04-25T10:59:21.228726+0000","last_fullsized":"2024-04-25T10:59:21.228726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:33.154184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228501+0000","last_change":"2024-04-25T10:59:18.640843+0000","last_active":"2024-04-25T10:59:21.228501+0000","last_peered":"2024-04-25T10:59:21.228501+0000","last_clean":"2024-04-25T10:59:21.228501+0000","last_became_active":"2024-04-25T10:59:18.640496+0000","last_became_peered":"2024-04-25T10:59:18.640496+0000","last_unstale":"2024-04-25T10:59:21.228501+0000","last_undegraded":"2024-04-25T10:59:21.228501+0000","last_fullsized":"2024-04-25T10:59:21.228501+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:05.044315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227754+0000","last_change":"2024-04-25T10:59:19.652337+0000","last_active":"2024-04-25T10:59:21.227754+0000","last_peered":"2024-04-25T10:59:21.227754+0000","last_clean":"2024-04-25T10:59:21.227754+0000","last_became_active":"2024-04-25T10:59:19.652008+0000","last_became_peered":"2024-04-25T10:59:19.652008+0000","last_unstale":"2024-04-25T10:59:21.227754+0000","last_undegraded":"2024-04-25T10:59:21.227754+0000","last_fullsized":"2024-04-25T10:59:21.227754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:32:26.553467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228556+0000","last_change":"2024-04-25T10:59:18.643508+0000","last_active":"2024-04-25T10:59:21.228556+0000","last_peered":"2024-04-25T10:59:21.228556+0000","last_clean":"2024-04-25T10:59:21.228556+0000","last_became_active":"2024-04-25T10:59:18.643186+0000","last_became_peered":"2024-04-25T10:59:18.643186+0000","last_unstale":"2024-04-25T10:59:21.228556+0000","last_undegraded":"2024-04-25T10:59:21.228556+0000","last_fullsized":"2024-04-25T10:59:21.228556+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:35.341389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228500+0000","last_change":"2024-04-25T10:59:19.640778+0000","last_active":"2024-04-25T10:59:21.228500+0000","last_peered":"2024-04-25T10:59:21.228500+0000","last_clean":"2024-04-25T10:59:21.228500+0000","last_became_active":"2024-04-25T10:59:19.640479+0000","last_became_peered":"2024-04-25T10:59:19.640479+0000","last_unstale":"2024-04-25T10:59:21.228500+0000","last_undegraded":"2024-04-25T10:59:21.228500+0000","last_fullsized":"2024-04-25T10:59:21.228500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:52.334437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228246+0000","last_change":"2024-04-25T10:59:18.635569+0000","last_active":"2024-04-25T10:59:21.228246+0000","last_peered":"2024-04-25T10:59:21.228246+0000","last_clean":"2024-04-25T10:59:21.228246+0000","last_became_active":"2024-04-25T10:59:18.635303+0000","last_became_peered":"2024-04-25T10:59:18.635303+0000","last_unstale":"2024-04-25T10:59:21.228246+0000","last_undegraded":"2024-04-25T10:59:21.228246+0000","last_fullsized":"2024-04-25T10:59:21.228246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:46.279987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":790528,"data_stored":655584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":196608,"data_stored":64724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":66246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":798720,"data_stored":655640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:59:22.628 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:59:22.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.769+0000 7f18cadf3640 1 Processor -- start 2024-04-25T10:59:22.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.769+0000 7f18cadf3640 1 -- start start 2024-04-25T10:59:22.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.771+0000 7f18cadf3640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18c415df80 0x7f18c4160fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:22.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.771+0000 7f18cadf3640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18c41615f0 con 0x7f18c40b8290 2024-04-25T10:59:22.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.771+0000 7f18cadf3640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18c4164020 con 0x7f18c4164b00 2024-04-25T10:59:22.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.771+0000 7f18cadf3640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f18c4164670 con 0x7f18c415df80 2024-04-25T10:59:22.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.771+0000 7f18c3fff640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f18c40b8290 0x7f18c4160890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:57138/0 (socket says 172.21.15.69:57138) 2024-04-25T10:59:22.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.771+0000 7f18c3fff640 1 -- 172.21.15.69:0/1045417127 learned_addr learned my addr 172.21.15.69:0/1045417127 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:22.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.771+0000 7f18c9390640 1 --2- 172.21.15.69:0/1045417127 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18c415df80 0x7f18c4160fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:22.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.771+0000 7f18c37fe640 1 -- 172.21.15.69:0/1045417127 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2094999990 0 0) 0x7f18c41615f0 con 0x7f18c40b8290 2024-04-25T10:59:22.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.771+0000 7f18c37fe640 1 -- 172.21.15.69:0/1045417127 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f18ac003520 con 0x7f18c40b8290 2024-04-25T10:59:22.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.771+0000 7f18c37fe640 1 -- 172.21.15.69:0/1045417127 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1536057142 0 0) 0x7f18ac003520 con 0x7f18c40b8290 2024-04-25T10:59:22.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.771+0000 7f18c37fe640 1 -- 172.21.15.69:0/1045417127 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18c415df80 msgr2=0x7f18c4160fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:22.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.771+0000 7f18c37fe640 1 --2- 172.21.15.69:0/1045417127 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18c415df80 0x7f18c4160fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:22.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.771+0000 7f18c37fe640 1 -- 172.21.15.69:0/1045417127 >> v1:172.21.15.146:6789/0 conn(0x7f18c4164b00 legacy=0x7f18c40b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:22.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.771+0000 7f18c37fe640 1 -- 172.21.15.69:0/1045417127 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c4164300 con 0x7f18c40b8290 2024-04-25T10:59:22.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.771+0000 7f18c37fe640 1 -- 172.21.15.69:0/1045417127 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7f18b8001670 con 0x7f18c40b8290 2024-04-25T10:59:22.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.771+0000 7f18c37fe640 1 -- 172.21.15.69:0/1045417127 >> v1:172.21.15.69:6789/0 conn(0x7f18c40b8290 legacy=0x7f18c4160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:22.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.772+0000 7f18c37fe640 1 --2- 172.21.15.69:0/1045417127 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18c415df80 0x7f18ac0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:22.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.772+0000 7f18c37fe640 1 --2- 172.21.15.69:0/1045417127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18ac003f60 0x7f18ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:22.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.772+0000 7f18c37fe640 1 --2- 172.21.15.69:0/1045417127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18ac004d40 0x7f18ac005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:22.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.772+0000 7f18c37fe640 1 -- 172.21.15.69:0/1045417127 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f18c4164670 con 0x7f18ac004d40 2024-04-25T10:59:22.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.772+0000 7f18c37fe640 1 -- 172.21.15.69:0/1045417127 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac00bb50 con 0x7f18ac003f60 2024-04-25T10:59:22.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.772+0000 7f18c37fe640 1 -- 172.21.15.69:0/1045417127 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f18ac00bd50 con 0x7f18c415df80 2024-04-25T10:59:22.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.772+0000 7f18c9390640 1 --2- 172.21.15.69:0/1045417127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18ac004d40 0x7f18ac005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:22.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.772+0000 7f18c8b8f640 1 --2- 172.21.15.69:0/1045417127 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18c415df80 0x7f18ac0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:22.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.772+0000 7f18c3fff640 1 --2- 172.21.15.69:0/1045417127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18ac003f60 0x7f18ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:22.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.772+0000 7f18c9390640 1 -- 172.21.15.69:0/1045417127 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18c415df80 msgr2=0x7f18ac0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:22.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.772+0000 7f18c9390640 1 --2- 172.21.15.69:0/1045417127 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18c415df80 0x7f18ac0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:22.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.772+0000 7f18c9390640 1 -- 172.21.15.69:0/1045417127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18ac003f60 msgr2=0x7f18ac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:22.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.772+0000 7f18c9390640 1 --2- 172.21.15.69:0/1045417127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18ac003f60 0x7f18ac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:22.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.772+0000 7f18c9390640 1 -- 172.21.15.69:0/1045417127 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18ac003520 con 0x7f18ac004d40 2024-04-25T10:59:22.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.773+0000 7f18c9390640 1 --2- 172.21.15.69:0/1045417127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18ac004d40 0x7f18ac005150 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f18c40d5bb0 tx=0x7f18ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ee053ca37ee4ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:22.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.773+0000 7f18c37fe640 1 -- 172.21.15.69:0/1045417127 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc009c10 con 0x7f18ac004d40 2024-04-25T10:59:22.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.773+0000 7f18c37fe640 1 -- 172.21.15.69:0/1045417127 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18bc036070 con 0x7f18ac004d40 2024-04-25T10:59:22.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.773+0000 7f18c37fe640 1 -- 172.21.15.69:0/1045417127 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc0317f0 con 0x7f18ac004d40 2024-04-25T10:59:22.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.773+0000 7f18cadf3640 1 -- 172.21.15.69:0/1045417127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18ac004d40 msgr2=0x7f18ac005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:22.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.773+0000 7f18cadf3640 1 --2- 172.21.15.69:0/1045417127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18ac004d40 0x7f18ac005150 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f18c40d5bb0 tx=0x7f18ac005830 comp rx=0 tx=0).stop 2024-04-25T10:59:22.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.773+0000 7f18c9390640 1 -- 172.21.15.69:0/1045417127 reap_dead start 2024-04-25T10:59:22.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.773+0000 7f18cadf3640 1 -- 172.21.15.69:0/1045417127 shutdown_connections 2024-04-25T10:59:22.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.773+0000 7f18cadf3640 1 -- 172.21.15.69:0/1045417127 >> 172.21.15.69:0/1045417127 conn(0x7f18c40d2bc0 msgr2=0x7f18c40d2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:22.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.773+0000 7f18cadf3640 1 -- 172.21.15.69:0/1045417127 shutdown_connections 2024-04-25T10:59:22.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.773+0000 7f18cadf3640 1 -- 172.21.15.69:0/1045417127 wait complete. 2024-04-25T10:59:22.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.775+0000 7f18cadf3640 1 Processor -- start 2024-04-25T10:59:22.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.775+0000 7f18cadf3640 1 -- start start 2024-04-25T10:59:22.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.777+0000 7f18cadf3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18c4151270 0x7f18c40d1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:22.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.777+0000 7f18cadf3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18c4155eb0 0x7f18c41562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:22.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.777+0000 7f18cadf3640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18c41f9820 0x7f18c41f9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:22.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.777+0000 7f18cadf3640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f18c4161330 con 0x7f18c4151270 2024-04-25T10:59:22.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.777+0000 7f18cadf3640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f18c41611b0 con 0x7f18c4155eb0 2024-04-25T10:59:22.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.777+0000 7f18cadf3640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f18c40b4880 con 0x7f18c41f9820 2024-04-25T10:59:22.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.777+0000 7f18c8b8f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18c4151270 0x7f18c40d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:22.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.777+0000 7f18c8b8f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18c4151270 0x7f18c40d1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41192/0 (socket says 172.21.15.69:41192) 2024-04-25T10:59:22.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.777+0000 7f18c8b8f640 1 -- 172.21.15.69:0/3786834926 learned_addr learned my addr 172.21.15.69:0/3786834926 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:22.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.777+0000 7f18c3fff640 1 --2- 172.21.15.69:0/3786834926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18c4155eb0 0x7f18c41562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:22.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.777+0000 7f18c9390640 1 --2- 172.21.15.69:0/3786834926 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18c41f9820 0x7f18c41f9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:22.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.777+0000 7f18c8b8f640 1 -- 172.21.15.69:0/3786834926 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18c41f9820 msgr2=0x7f18c41f9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:22.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.777+0000 7f18c8b8f640 1 --2- 172.21.15.69:0/3786834926 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18c41f9820 0x7f18c41f9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:22.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.777+0000 7f18c8b8f640 1 -- 172.21.15.69:0/3786834926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18c4155eb0 msgr2=0x7f18c41562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:22.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.777+0000 7f18c8b8f640 1 --2- 172.21.15.69:0/3786834926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18c4155eb0 0x7f18c41562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:22.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.777+0000 7f18c8b8f640 1 -- 172.21.15.69:0/3786834926 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c4156800 con 0x7f18c4151270 2024-04-25T10:59:22.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.777+0000 7f18c8b8f640 1 --2- 172.21.15.69:0/3786834926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18c4151270 0x7f18c40d1110 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f18b400f850 tx=0x7f18b400fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:22.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.777+0000 7f18c17fa640 1 -- 172.21.15.69:0/3786834926 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b400fe40 con 0x7f18c4151270 2024-04-25T10:59:22.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.777+0000 7f18c17fa640 1 -- 172.21.15.69:0/3786834926 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18b4018070 con 0x7f18c4151270 2024-04-25T10:59:22.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.777+0000 7f18cadf3640 1 -- 172.21.15.69:0/3786834926 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18c4153670 con 0x7f18c4151270 2024-04-25T10:59:22.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.777+0000 7f18c17fa640 1 -- 172.21.15.69:0/3786834926 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b400d730 con 0x7f18c4151270 2024-04-25T10:59:22.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.777+0000 7f18cadf3640 1 -- 172.21.15.69:0/3786834926 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18c4158e00 con 0x7f18c4151270 2024-04-25T10:59:22.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.778+0000 7f18cadf3640 1 -- 172.21.15.69:0/3786834926 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18c4158940 con 0x7f18c4151270 2024-04-25T10:59:22.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.779+0000 7f18c17fa640 1 -- 172.21.15.69:0/3786834926 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f18b400d8d0 con 0x7f18c4151270 2024-04-25T10:59:22.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.779+0000 7f18c17fa640 1 --2- 172.21.15.69:0/3786834926 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f18b00744b0 0x7f18b00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:22.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.780+0000 7f18c17fa640 1 -- 172.21.15.69:0/3786834926 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f18b4095000 con 0x7f18c4151270 2024-04-25T10:59:22.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.780+0000 7f18c3fff640 1 --2- 172.21.15.69:0/3786834926 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f18b00744b0 0x7f18b00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:22.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.780+0000 7f18c3fff640 1 --2- 172.21.15.69:0/3786834926 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f18b00744b0 0x7f18b00769c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f18c41525a0 tx=0x7f18b8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:22.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.784+0000 7f18c17fa640 1 -- 172.21.15.69:0/3786834926 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f18c4158940 con 0x7f18c4151270 2024-04-25T10:59:22.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.956+0000 7f18cadf3640 1 -- 172.21.15.69:0/3786834926 --> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f18c4151b50 con 0x7f18b00744b0 2024-04-25T10:59:22.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.962+0000 7f18c17fa640 1 -- 172.21.15.69:0/3786834926 <== mgr.4107 v2:172.21.15.69:6832/1107421267 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251936 (secure 0 0 0) 0x7f18b80076b0 con 0x7f18b00744b0 2024-04-25T10:59:22.965 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:22.966 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-04-25T10:59:22.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.965+0000 7f18cadf3640 1 -- 172.21.15.69:0/3786834926 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f18b00744b0 msgr2=0x7f18b00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:22.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.965+0000 7f18cadf3640 1 --2- 172.21.15.69:0/3786834926 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f18b00744b0 0x7f18b00769c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f18c41525a0 tx=0x7f18b8009040 comp rx=0 tx=0).stop 2024-04-25T10:59:22.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.965+0000 7f18cadf3640 1 -- 172.21.15.69:0/3786834926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18c4151270 msgr2=0x7f18c40d1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:22.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.965+0000 7f18cadf3640 1 --2- 172.21.15.69:0/3786834926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18c4151270 0x7f18c40d1110 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f18b400f850 tx=0x7f18b400fd10 comp rx=0 tx=0).stop 2024-04-25T10:59:22.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.965+0000 7f18cadf3640 1 -- 172.21.15.69:0/3786834926 shutdown_connections 2024-04-25T10:59:22.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.965+0000 7f18cadf3640 1 --2- 172.21.15.69:0/3786834926 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f18b00744b0 0x7f18b00769c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:22.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.965+0000 7f18cadf3640 1 --2- 172.21.15.69:0/3786834926 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18c41f9820 0x7f18c41f9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:22.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.965+0000 7f18cadf3640 1 --2- 172.21.15.69:0/3786834926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18c4155eb0 0x7f18c41562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:22.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.965+0000 7f18cadf3640 1 --2- 172.21.15.69:0/3786834926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18c4151270 0x7f18c40d1110 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:22.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.965+0000 7f18cadf3640 1 -- 172.21.15.69:0/3786834926 >> 172.21.15.69:0/3786834926 conn(0x7f18c40d2bc0 msgr2=0x7f18c415f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:22.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.965+0000 7f18cadf3640 1 -- 172.21.15.69:0/3786834926 shutdown_connections 2024-04-25T10:59:22.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:22.965+0000 7f18cadf3640 1 -- 172.21.15.69:0/3786834926 wait complete. 2024-04-25T10:59:22.977 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-25T10:59:22.406342+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":180,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218948,"kb_used_data":2764,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755772,"statfs":{"total":773094113280,"available":772869910528,"internally_reserved":0,"allocated":2830336,"data_stored":1721528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.794625"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228976+0000","last_change":"2024-04-25T10:59:19.640957+0000","last_active":"2024-04-25T10:59:21.228976+0000","last_peered":"2024-04-25T10:59:21.228976+0000","last_clean":"2024-04-25T10:59:21.228976+0000","last_became_active":"2024-04-25T10:59:19.640610+0000","last_became_peered":"2024-04-25T10:59:19.640610+0000","last_unstale":"2024-04-25T10:59:21.228976+0000","last_undegraded":"2024-04-25T10:59:21.228976+0000","last_fullsized":"2024-04-25T10:59:21.228976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:42.867755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636604+0000","last_change":"2024-04-25T10:59:19.644473+0000","last_active":"2024-04-25T10:59:20.636604+0000","last_peered":"2024-04-25T10:59:20.636604+0000","last_clean":"2024-04-25T10:59:20.636604+0000","last_became_active":"2024-04-25T10:59:19.644151+0000","last_became_peered":"2024-04-25T10:59:19.644151+0000","last_unstale":"2024-04-25T10:59:20.636604+0000","last_undegraded":"2024-04-25T10:59:20.636604+0000","last_fullsized":"2024-04-25T10:59:20.636604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:07.522883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227761+0000","last_change":"2024-04-25T10:59:19.630984+0000","last_active":"2024-04-25T10:59:21.227761+0000","last_peered":"2024-04-25T10:59:21.227761+0000","last_clean":"2024-04-25T10:59:21.227761+0000","last_became_active":"2024-04-25T10:59:19.630699+0000","last_became_peered":"2024-04-25T10:59:19.630699+0000","last_unstale":"2024-04-25T10:59:21.227761+0000","last_undegraded":"2024-04-25T10:59:21.227761+0000","last_fullsized":"2024-04-25T10:59:21.227761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:59.213210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228437+0000","last_change":"2024-04-25T10:59:19.649708+0000","last_active":"2024-04-25T10:59:21.228437+0000","last_peered":"2024-04-25T10:59:21.228437+0000","last_clean":"2024-04-25T10:59:21.228437+0000","last_became_active":"2024-04-25T10:59:19.649262+0000","last_became_peered":"2024-04-25T10:59:19.649262+0000","last_unstale":"2024-04-25T10:59:21.228437+0000","last_undegraded":"2024-04-25T10:59:21.228437+0000","last_fullsized":"2024-04-25T10:59:21.228437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:52.334437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228519+0000","last_change":"2024-04-25T10:59:19.642490+0000","last_active":"2024-04-25T10:59:21.228519+0000","last_peered":"2024-04-25T10:59:21.228519+0000","last_clean":"2024-04-25T10:59:21.228519+0000","last_became_active":"2024-04-25T10:59:19.642122+0000","last_became_peered":"2024-04-25T10:59:19.642122+0000","last_unstale":"2024-04-25T10:59:21.228519+0000","last_undegraded":"2024-04-25T10:59:21.228519+0000","last_fullsized":"2024-04-25T10:59:21.228519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:14.189727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228892+0000","last_change":"2024-04-25T10:59:19.651061+0000","last_active":"2024-04-25T10:59:21.228892+0000","last_peered":"2024-04-25T10:59:21.228892+0000","last_clean":"2024-04-25T10:59:21.228892+0000","last_became_active":"2024-04-25T10:59:19.650710+0000","last_became_peered":"2024-04-25T10:59:19.650710+0000","last_unstale":"2024-04-25T10:59:21.228892+0000","last_undegraded":"2024-04-25T10:59:21.228892+0000","last_fullsized":"2024-04-25T10:59:21.228892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:45.328749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227817+0000","last_change":"2024-04-25T10:59:19.646862+0000","last_active":"2024-04-25T10:59:21.227817+0000","last_peered":"2024-04-25T10:59:21.227817+0000","last_clean":"2024-04-25T10:59:21.227817+0000","last_became_active":"2024-04-25T10:59:19.646553+0000","last_became_peered":"2024-04-25T10:59:19.646553+0000","last_unstale":"2024-04-25T10:59:21.227817+0000","last_undegraded":"2024-04-25T10:59:21.227817+0000","last_fullsized":"2024-04-25T10:59:21.227817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:52.334437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227508+0000","last_change":"2024-04-25T10:59:19.648086+0000","last_active":"2024-04-25T10:59:21.227508+0000","last_peered":"2024-04-25T10:59:21.227508+0000","last_clean":"2024-04-25T10:59:21.227508+0000","last_became_active":"2024-04-25T10:59:19.647702+0000","last_became_peered":"2024-04-25T10:59:19.647702+0000","last_unstale":"2024-04-25T10:59:21.227508+0000","last_undegraded":"2024-04-25T10:59:21.227508+0000","last_fullsized":"2024-04-25T10:59:21.227508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:59.213210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228983+0000","last_change":"2024-04-25T10:59:19.655202+0000","last_active":"2024-04-25T10:59:21.228983+0000","last_peered":"2024-04-25T10:59:21.228983+0000","last_clean":"2024-04-25T10:59:21.228983+0000","last_became_active":"2024-04-25T10:59:19.654871+0000","last_became_peered":"2024-04-25T10:59:19.654871+0000","last_unstale":"2024-04-25T10:59:21.228983+0000","last_undegraded":"2024-04-25T10:59:21.228983+0000","last_fullsized":"2024-04-25T10:59:21.228983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:50.893634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638094+0000","last_change":"2024-04-25T10:59:19.651928+0000","last_active":"2024-04-25T10:59:20.638094+0000","last_peered":"2024-04-25T10:59:20.638094+0000","last_clean":"2024-04-25T10:59:20.638094+0000","last_became_active":"2024-04-25T10:59:19.651618+0000","last_became_peered":"2024-04-25T10:59:19.651618+0000","last_unstale":"2024-04-25T10:59:20.638094+0000","last_undegraded":"2024-04-25T10:59:20.638094+0000","last_fullsized":"2024-04-25T10:59:20.638094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:32:26.553467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638187+0000","last_change":"2024-04-25T10:59:19.652810+0000","last_active":"2024-04-25T10:59:20.638187+0000","last_peered":"2024-04-25T10:59:20.638187+0000","last_clean":"2024-04-25T10:59:20.638187+0000","last_became_active":"2024-04-25T10:59:19.652118+0000","last_became_peered":"2024-04-25T10:59:19.652118+0000","last_unstale":"2024-04-25T10:59:20.638187+0000","last_undegraded":"2024-04-25T10:59:20.638187+0000","last_fullsized":"2024-04-25T10:59:20.638187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:59.213210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638421+0000","last_change":"2024-04-25T10:59:19.645548+0000","last_active":"2024-04-25T10:59:20.638421+0000","last_peered":"2024-04-25T10:59:20.638421+0000","last_clean":"2024-04-25T10:59:20.638421+0000","last_became_active":"2024-04-25T10:59:19.645231+0000","last_became_peered":"2024-04-25T10:59:19.645231+0000","last_unstale":"2024-04-25T10:59:20.638421+0000","last_undegraded":"2024-04-25T10:59:20.638421+0000","last_fullsized":"2024-04-25T10:59:20.638421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:59.512598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227723+0000","last_change":"2024-04-25T10:59:19.640736+0000","last_active":"2024-04-25T10:59:21.227723+0000","last_peered":"2024-04-25T10:59:21.227723+0000","last_clean":"2024-04-25T10:59:21.227723+0000","last_became_active":"2024-04-25T10:59:19.640448+0000","last_became_peered":"2024-04-25T10:59:19.640448+0000","last_unstale":"2024-04-25T10:59:21.227723+0000","last_undegraded":"2024-04-25T10:59:21.227723+0000","last_fullsized":"2024-04-25T10:59:21.227723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:15.132862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228215+0000","last_change":"2024-04-25T10:59:19.634929+0000","last_active":"2024-04-25T10:59:21.228215+0000","last_peered":"2024-04-25T10:59:21.228215+0000","last_clean":"2024-04-25T10:59:21.228215+0000","last_became_active":"2024-04-25T10:59:19.634640+0000","last_became_peered":"2024-04-25T10:59:19.634640+0000","last_unstale":"2024-04-25T10:59:21.228215+0000","last_undegraded":"2024-04-25T10:59:21.228215+0000","last_fullsized":"2024-04-25T10:59:21.228215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:49.314956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636375+0000","last_change":"2024-04-25T10:59:19.648597+0000","last_active":"2024-04-25T10:59:20.636375+0000","last_peered":"2024-04-25T10:59:20.636375+0000","last_clean":"2024-04-25T10:59:20.636375+0000","last_became_active":"2024-04-25T10:59:19.648280+0000","last_became_peered":"2024-04-25T10:59:19.648280+0000","last_unstale":"2024-04-25T10:59:20.636375+0000","last_undegraded":"2024-04-25T10:59:20.636375+0000","last_fullsized":"2024-04-25T10:59:20.636375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:58.824966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228034+0000","last_change":"2024-04-25T10:59:19.647183+0000","last_active":"2024-04-25T10:59:21.228034+0000","last_peered":"2024-04-25T10:59:21.228034+0000","last_clean":"2024-04-25T10:59:21.228034+0000","last_became_active":"2024-04-25T10:59:19.646563+0000","last_became_peered":"2024-04-25T10:59:19.646563+0000","last_unstale":"2024-04-25T10:59:21.228034+0000","last_undegraded":"2024-04-25T10:59:21.228034+0000","last_fullsized":"2024-04-25T10:59:21.228034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:57.344787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228532+0000","last_change":"2024-04-25T10:59:19.649207+0000","last_active":"2024-04-25T10:59:21.228532+0000","last_peered":"2024-04-25T10:59:21.228532+0000","last_clean":"2024-04-25T10:59:21.228532+0000","last_became_active":"2024-04-25T10:59:19.647280+0000","last_became_peered":"2024-04-25T10:59:19.647280+0000","last_unstale":"2024-04-25T10:59:21.228532+0000","last_undegraded":"2024-04-25T10:59:21.228532+0000","last_fullsized":"2024-04-25T10:59:21.228532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:32.457404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.185793+0000","last_change":"2024-04-25T10:59:19.640810+0000","last_active":"2024-04-25T10:59:21.185793+0000","last_peered":"2024-04-25T10:59:21.185793+0000","last_clean":"2024-04-25T10:59:21.185793+0000","last_became_active":"2024-04-25T10:59:19.640524+0000","last_became_peered":"2024-04-25T10:59:19.640524+0000","last_unstale":"2024-04-25T10:59:21.185793+0000","last_undegraded":"2024-04-25T10:59:21.185793+0000","last_fullsized":"2024-04-25T10:59:21.185793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:58.824966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228703+0000","last_change":"2024-04-25T10:59:19.641698+0000","last_active":"2024-04-25T10:59:21.228703+0000","last_peered":"2024-04-25T10:59:21.228703+0000","last_clean":"2024-04-25T10:59:21.228703+0000","last_became_active":"2024-04-25T10:59:19.640996+0000","last_became_peered":"2024-04-25T10:59:19.640996+0000","last_unstale":"2024-04-25T10:59:21.228703+0000","last_undegraded":"2024-04-25T10:59:21.228703+0000","last_fullsized":"2024-04-25T10:59:21.228703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:33.154184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.639226+0000","last_change":"2024-04-25T10:59:19.654176+0000","last_active":"2024-04-25T10:59:20.639226+0000","last_peered":"2024-04-25T10:59:20.639226+0000","last_clean":"2024-04-25T10:59:20.639226+0000","last_became_active":"2024-04-25T10:59:19.653175+0000","last_became_peered":"2024-04-25T10:59:19.653175+0000","last_unstale":"2024-04-25T10:59:20.639226+0000","last_undegraded":"2024-04-25T10:59:20.639226+0000","last_fullsized":"2024-04-25T10:59:20.639226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:33.050282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227935+0000","last_change":"2024-04-25T10:59:19.642959+0000","last_active":"2024-04-25T10:59:21.227935+0000","last_peered":"2024-04-25T10:59:21.227935+0000","last_clean":"2024-04-25T10:59:21.227935+0000","last_became_active":"2024-04-25T10:59:19.642671+0000","last_became_peered":"2024-04-25T10:59:19.642671+0000","last_unstale":"2024-04-25T10:59:21.227935+0000","last_undegraded":"2024-04-25T10:59:21.227935+0000","last_fullsized":"2024-04-25T10:59:21.227935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:15.132862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228503+0000","last_change":"2024-04-25T10:59:19.658039+0000","last_active":"2024-04-25T10:59:21.228503+0000","last_peered":"2024-04-25T10:59:21.228503+0000","last_clean":"2024-04-25T10:59:21.228503+0000","last_became_active":"2024-04-25T10:59:19.657194+0000","last_became_peered":"2024-04-25T10:59:19.657194+0000","last_unstale":"2024-04-25T10:59:21.228503+0000","last_undegraded":"2024-04-25T10:59:21.228503+0000","last_fullsized":"2024-04-25T10:59:21.228503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:19.485287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227782+0000","last_change":"2024-04-25T10:59:19.645451+0000","last_active":"2024-04-25T10:59:21.227782+0000","last_peered":"2024-04-25T10:59:21.227782+0000","last_clean":"2024-04-25T10:59:21.227782+0000","last_became_active":"2024-04-25T10:59:19.645121+0000","last_became_peered":"2024-04-25T10:59:19.645121+0000","last_unstale":"2024-04-25T10:59:21.227782+0000","last_undegraded":"2024-04-25T10:59:21.227782+0000","last_fullsized":"2024-04-25T10:59:21.227782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:07.522883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228328+0000","last_change":"2024-04-25T10:59:19.646916+0000","last_active":"2024-04-25T10:59:21.228328+0000","last_peered":"2024-04-25T10:59:21.228328+0000","last_clean":"2024-04-25T10:59:21.228328+0000","last_became_active":"2024-04-25T10:59:19.646228+0000","last_became_peered":"2024-04-25T10:59:19.646228+0000","last_unstale":"2024-04-25T10:59:21.228328+0000","last_undegraded":"2024-04-25T10:59:21.228328+0000","last_fullsized":"2024-04-25T10:59:21.228328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:17.431757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228328+0000","last_change":"2024-04-25T10:59:19.644795+0000","last_active":"2024-04-25T10:59:21.228328+0000","last_peered":"2024-04-25T10:59:21.228328+0000","last_clean":"2024-04-25T10:59:21.228328+0000","last_became_active":"2024-04-25T10:59:19.644503+0000","last_became_peered":"2024-04-25T10:59:19.644503+0000","last_unstale":"2024-04-25T10:59:21.228328+0000","last_undegraded":"2024-04-25T10:59:21.228328+0000","last_fullsized":"2024-04-25T10:59:21.228328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:17.431757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638291+0000","last_change":"2024-04-25T10:59:19.650630+0000","last_active":"2024-04-25T10:59:20.638291+0000","last_peered":"2024-04-25T10:59:20.638291+0000","last_clean":"2024-04-25T10:59:20.638291+0000","last_became_active":"2024-04-25T10:59:19.650312+0000","last_became_peered":"2024-04-25T10:59:19.650312+0000","last_unstale":"2024-04-25T10:59:20.638291+0000","last_undegraded":"2024-04-25T10:59:20.638291+0000","last_fullsized":"2024-04-25T10:59:20.638291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:58.824966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636446+0000","last_change":"2024-04-25T10:59:19.641641+0000","last_active":"2024-04-25T10:59:20.636446+0000","last_peered":"2024-04-25T10:59:20.636446+0000","last_clean":"2024-04-25T10:59:20.636446+0000","last_became_active":"2024-04-25T10:59:19.641339+0000","last_became_peered":"2024-04-25T10:59:19.641339+0000","last_unstale":"2024-04-25T10:59:20.636446+0000","last_undegraded":"2024-04-25T10:59:20.636446+0000","last_fullsized":"2024-04-25T10:59:20.636446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:52.815596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228173+0000","last_change":"2024-04-25T10:59:19.642824+0000","last_active":"2024-04-25T10:59:21.228173+0000","last_peered":"2024-04-25T10:59:21.228173+0000","last_clean":"2024-04-25T10:59:21.228173+0000","last_became_active":"2024-04-25T10:59:19.641152+0000","last_became_peered":"2024-04-25T10:59:19.641152+0000","last_unstale":"2024-04-25T10:59:21.228173+0000","last_undegraded":"2024-04-25T10:59:21.228173+0000","last_fullsized":"2024-04-25T10:59:21.228173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:49.314956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228117+0000","last_change":"2024-04-25T10:59:19.629558+0000","last_active":"2024-04-25T10:59:21.228117+0000","last_peered":"2024-04-25T10:59:21.228117+0000","last_clean":"2024-04-25T10:59:21.228117+0000","last_became_active":"2024-04-25T10:59:19.628864+0000","last_became_peered":"2024-04-25T10:59:19.628864+0000","last_unstale":"2024-04-25T10:59:21.228117+0000","last_undegraded":"2024-04-25T10:59:21.228117+0000","last_fullsized":"2024-04-25T10:59:21.228117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:07.522883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228910+0000","last_change":"2024-04-25T10:59:19.648576+0000","last_active":"2024-04-25T10:59:21.228910+0000","last_peered":"2024-04-25T10:59:21.228910+0000","last_clean":"2024-04-25T10:59:21.228910+0000","last_became_active":"2024-04-25T10:59:19.647435+0000","last_became_peered":"2024-04-25T10:59:19.647435+0000","last_unstale":"2024-04-25T10:59:21.228910+0000","last_undegraded":"2024-04-25T10:59:21.228910+0000","last_fullsized":"2024-04-25T10:59:21.228910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:42.867755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.185603+0000","last_change":"2024-04-25T10:59:19.649031+0000","last_active":"2024-04-25T10:59:21.185603+0000","last_peered":"2024-04-25T10:59:21.185603+0000","last_clean":"2024-04-25T10:59:21.185603+0000","last_became_active":"2024-04-25T10:59:19.648698+0000","last_became_peered":"2024-04-25T10:59:19.648698+0000","last_unstale":"2024-04-25T10:59:21.185603+0000","last_undegraded":"2024-04-25T10:59:21.185603+0000","last_fullsized":"2024-04-25T10:59:21.185603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:59.213210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638942+0000","last_change":"2024-04-25T10:59:19.643547+0000","last_active":"2024-04-25T10:59:20.638942+0000","last_peered":"2024-04-25T10:59:20.638942+0000","last_clean":"2024-04-25T10:59:20.638942+0000","last_became_active":"2024-04-25T10:59:19.642802+0000","last_became_peered":"2024-04-25T10:59:19.642802+0000","last_unstale":"2024-04-25T10:59:20.638942+0000","last_undegraded":"2024-04-25T10:59:20.638942+0000","last_fullsized":"2024-04-25T10:59:20.638942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:15.132862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638161+0000","last_change":"2024-04-25T10:59:19.638090+0000","last_active":"2024-04-25T10:59:20.638161+0000","last_peered":"2024-04-25T10:59:20.638161+0000","last_clean":"2024-04-25T10:59:20.638161+0000","last_became_active":"2024-04-25T10:59:19.637745+0000","last_became_peered":"2024-04-25T10:59:19.637745+0000","last_unstale":"2024-04-25T10:59:20.638161+0000","last_undegraded":"2024-04-25T10:59:20.638161+0000","last_fullsized":"2024-04-25T10:59:20.638161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:25.274583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227963+0000","last_change":"2024-04-25T10:59:18.637294+0000","last_active":"2024-04-25T10:59:21.227963+0000","last_peered":"2024-04-25T10:59:21.227963+0000","last_clean":"2024-04-25T10:59:21.227963+0000","last_became_active":"2024-04-25T10:59:18.636963+0000","last_became_peered":"2024-04-25T10:59:18.636963+0000","last_unstale":"2024-04-25T10:59:21.227963+0000","last_undegraded":"2024-04-25T10:59:21.227963+0000","last_fullsized":"2024-04-25T10:59:21.227963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:18.298759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228183+0000","last_change":"2024-04-25T10:59:19.638014+0000","last_active":"2024-04-25T10:59:21.228183+0000","last_peered":"2024-04-25T10:59:21.228183+0000","last_clean":"2024-04-25T10:59:21.228183+0000","last_became_active":"2024-04-25T10:59:19.637471+0000","last_became_peered":"2024-04-25T10:59:19.637471+0000","last_unstale":"2024-04-25T10:59:21.228183+0000","last_undegraded":"2024-04-25T10:59:21.228183+0000","last_fullsized":"2024-04-25T10:59:21.228183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:33.050282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638880+0000","last_change":"2024-04-25T10:59:18.634982+0000","last_active":"2024-04-25T10:59:20.638880+0000","last_peered":"2024-04-25T10:59:20.638880+0000","last_clean":"2024-04-25T10:59:20.638880+0000","last_became_active":"2024-04-25T10:59:18.634658+0000","last_became_peered":"2024-04-25T10:59:18.634658+0000","last_unstale":"2024-04-25T10:59:20.638880+0000","last_undegraded":"2024-04-25T10:59:20.638880+0000","last_fullsized":"2024-04-25T10:59:20.638880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:35.341389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227940+0000","last_change":"2024-04-25T10:59:18.632617+0000","last_active":"2024-04-25T10:59:21.227940+0000","last_peered":"2024-04-25T10:59:21.227940+0000","last_clean":"2024-04-25T10:59:21.227940+0000","last_became_active":"2024-04-25T10:59:18.632294+0000","last_became_peered":"2024-04-25T10:59:18.632294+0000","last_unstale":"2024-04-25T10:59:21.227940+0000","last_undegraded":"2024-04-25T10:59:21.227940+0000","last_fullsized":"2024-04-25T10:59:21.227940+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:46.279987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227575+0000","last_change":"2024-04-25T10:59:19.632333+0000","last_active":"2024-04-25T10:59:21.227575+0000","last_peered":"2024-04-25T10:59:21.227575+0000","last_clean":"2024-04-25T10:59:21.227575+0000","last_became_active":"2024-04-25T10:59:19.632042+0000","last_became_peered":"2024-04-25T10:59:19.632042+0000","last_unstale":"2024-04-25T10:59:21.227575+0000","last_undegraded":"2024-04-25T10:59:21.227575+0000","last_fullsized":"2024-04-25T10:59:21.227575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:58.824966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228184+0000","last_change":"2024-04-25T10:59:18.638103+0000","last_active":"2024-04-25T10:59:21.228184+0000","last_peered":"2024-04-25T10:59:21.228184+0000","last_clean":"2024-04-25T10:59:21.228184+0000","last_became_active":"2024-04-25T10:59:18.636931+0000","last_became_peered":"2024-04-25T10:59:18.636931+0000","last_unstale":"2024-04-25T10:59:21.228184+0000","last_undegraded":"2024-04-25T10:59:21.228184+0000","last_fullsized":"2024-04-25T10:59:21.228184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:15.754476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636374+0000","last_change":"2024-04-25T10:59:19.633809+0000","last_active":"2024-04-25T10:59:20.636374+0000","last_peered":"2024-04-25T10:59:20.636374+0000","last_clean":"2024-04-25T10:59:20.636374+0000","last_became_active":"2024-04-25T10:59:19.632787+0000","last_became_peered":"2024-04-25T10:59:19.632787+0000","last_unstale":"2024-04-25T10:59:20.636374+0000","last_undegraded":"2024-04-25T10:59:20.636374+0000","last_fullsized":"2024-04-25T10:59:20.636374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:59.213210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638419+0000","last_change":"2024-04-25T10:59:18.641813+0000","last_active":"2024-04-25T10:59:20.638419+0000","last_peered":"2024-04-25T10:59:20.638419+0000","last_clean":"2024-04-25T10:59:20.638419+0000","last_became_active":"2024-04-25T10:59:18.641127+0000","last_became_peered":"2024-04-25T10:59:18.641127+0000","last_unstale":"2024-04-25T10:59:20.638419+0000","last_undegraded":"2024-04-25T10:59:20.638419+0000","last_fullsized":"2024-04-25T10:59:20.638419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:28.594312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228699+0000","last_change":"2024-04-25T10:59:19.645823+0000","last_active":"2024-04-25T10:59:21.228699+0000","last_peered":"2024-04-25T10:59:21.228699+0000","last_clean":"2024-04-25T10:59:21.228699+0000","last_became_active":"2024-04-25T10:59:19.645500+0000","last_became_peered":"2024-04-25T10:59:19.645500+0000","last_unstale":"2024-04-25T10:59:21.228699+0000","last_undegraded":"2024-04-25T10:59:21.228699+0000","last_fullsized":"2024-04-25T10:59:21.228699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:00.311029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228172+0000","last_change":"2024-04-25T10:59:18.641002+0000","last_active":"2024-04-25T10:59:21.228172+0000","last_peered":"2024-04-25T10:59:21.228172+0000","last_clean":"2024-04-25T10:59:21.228172+0000","last_became_active":"2024-04-25T10:59:18.640589+0000","last_became_peered":"2024-04-25T10:59:18.640589+0000","last_unstale":"2024-04-25T10:59:21.228172+0000","last_undegraded":"2024-04-25T10:59:21.228172+0000","last_fullsized":"2024-04-25T10:59:21.228172+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:24.267022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227906+0000","last_change":"2024-04-25T10:59:19.652033+0000","last_active":"2024-04-25T10:59:21.227906+0000","last_peered":"2024-04-25T10:59:21.227906+0000","last_clean":"2024-04-25T10:59:21.227906+0000","last_became_active":"2024-04-25T10:59:19.651694+0000","last_became_peered":"2024-04-25T10:59:19.651694+0000","last_unstale":"2024-04-25T10:59:21.227906+0000","last_undegraded":"2024-04-25T10:59:21.227906+0000","last_fullsized":"2024-04-25T10:59:21.227906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:58.824966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636485+0000","last_change":"2024-04-25T10:59:18.638096+0000","last_active":"2024-04-25T10:59:20.636485+0000","last_peered":"2024-04-25T10:59:20.636485+0000","last_clean":"2024-04-25T10:59:20.636485+0000","last_became_active":"2024-04-25T10:59:18.637808+0000","last_became_peered":"2024-04-25T10:59:18.637808+0000","last_unstale":"2024-04-25T10:59:20.636485+0000","last_undegraded":"2024-04-25T10:59:20.636485+0000","last_fullsized":"2024-04-25T10:59:20.636485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:10.473815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638613+0000","last_change":"2024-04-25T10:59:19.638709+0000","last_active":"2024-04-25T10:59:20.638613+0000","last_peered":"2024-04-25T10:59:20.638613+0000","last_clean":"2024-04-25T10:59:20.638613+0000","last_became_active":"2024-04-25T10:59:19.638358+0000","last_became_peered":"2024-04-25T10:59:19.638358+0000","last_unstale":"2024-04-25T10:59:20.638613+0000","last_undegraded":"2024-04-25T10:59:20.638613+0000","last_fullsized":"2024-04-25T10:59:20.638613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:46.619782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.639204+0000","last_change":"2024-04-25T10:59:18.639824+0000","last_active":"2024-04-25T10:59:20.639204+0000","last_peered":"2024-04-25T10:59:20.639204+0000","last_clean":"2024-04-25T10:59:20.639204+0000","last_became_active":"2024-04-25T10:59:18.639316+0000","last_became_peered":"2024-04-25T10:59:18.639316+0000","last_unstale":"2024-04-25T10:59:20.639204+0000","last_undegraded":"2024-04-25T10:59:20.639204+0000","last_fullsized":"2024-04-25T10:59:20.639204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:15.754476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228064+0000","last_change":"2024-04-25T10:59:19.641380+0000","last_active":"2024-04-25T10:59:21.228064+0000","last_peered":"2024-04-25T10:59:21.228064+0000","last_clean":"2024-04-25T10:59:21.228064+0000","last_became_active":"2024-04-25T10:59:19.640580+0000","last_became_peered":"2024-04-25T10:59:19.640580+0000","last_unstale":"2024-04-25T10:59:21.228064+0000","last_undegraded":"2024-04-25T10:59:21.228064+0000","last_fullsized":"2024-04-25T10:59:21.228064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:19.485287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228226+0000","last_change":"2024-04-25T10:59:18.634324+0000","last_active":"2024-04-25T10:59:21.228226+0000","last_peered":"2024-04-25T10:59:21.228226+0000","last_clean":"2024-04-25T10:59:21.228226+0000","last_became_active":"2024-04-25T10:59:18.634049+0000","last_became_peered":"2024-04-25T10:59:18.634049+0000","last_unstale":"2024-04-25T10:59:21.228226+0000","last_undegraded":"2024-04-25T10:59:21.228226+0000","last_fullsized":"2024-04-25T10:59:21.228226+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:02.619679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228376+0000","last_change":"2024-04-25T10:59:19.643539+0000","last_active":"2024-04-25T10:59:21.228376+0000","last_peered":"2024-04-25T10:59:21.228376+0000","last_clean":"2024-04-25T10:59:21.228376+0000","last_became_active":"2024-04-25T10:59:19.643182+0000","last_became_peered":"2024-04-25T10:59:19.643182+0000","last_unstale":"2024-04-25T10:59:21.228376+0000","last_undegraded":"2024-04-25T10:59:21.228376+0000","last_fullsized":"2024-04-25T10:59:21.228376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:57.344787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.229036+0000","last_change":"2024-04-25T10:59:18.643936+0000","last_active":"2024-04-25T10:59:21.229036+0000","last_peered":"2024-04-25T10:59:21.229036+0000","last_clean":"2024-04-25T10:59:21.229036+0000","last_became_active":"2024-04-25T10:59:18.643576+0000","last_became_peered":"2024-04-25T10:59:18.643576+0000","last_unstale":"2024-04-25T10:59:21.229036+0000","last_undegraded":"2024-04-25T10:59:21.229036+0000","last_fullsized":"2024-04-25T10:59:21.229036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:58.505038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228718+0000","last_change":"2024-04-25T10:59:19.652528+0000","last_active":"2024-04-25T10:59:21.228718+0000","last_peered":"2024-04-25T10:59:21.228718+0000","last_clean":"2024-04-25T10:59:21.228718+0000","last_became_active":"2024-04-25T10:59:19.651277+0000","last_became_peered":"2024-04-25T10:59:19.651277+0000","last_unstale":"2024-04-25T10:59:21.228718+0000","last_undegraded":"2024-04-25T10:59:21.228718+0000","last_fullsized":"2024-04-25T10:59:21.228718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:45.328749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.637715+0000","last_change":"2024-04-25T10:59:18.630832+0000","last_active":"2024-04-25T10:59:20.637715+0000","last_peered":"2024-04-25T10:59:20.637715+0000","last_clean":"2024-04-25T10:59:20.637715+0000","last_became_active":"2024-04-25T10:59:18.630462+0000","last_became_peered":"2024-04-25T10:59:18.630462+0000","last_unstale":"2024-04-25T10:59:20.637715+0000","last_undegraded":"2024-04-25T10:59:20.637715+0000","last_fullsized":"2024-04-25T10:59:20.637715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:46.279987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.639078+0000","last_change":"2024-04-25T10:59:19.643805+0000","last_active":"2024-04-25T10:59:20.639078+0000","last_peered":"2024-04-25T10:59:20.639078+0000","last_clean":"2024-04-25T10:59:20.639078+0000","last_became_active":"2024-04-25T10:59:19.643222+0000","last_became_peered":"2024-04-25T10:59:19.643222+0000","last_unstale":"2024-04-25T10:59:20.639078+0000","last_undegraded":"2024-04-25T10:59:20.639078+0000","last_fullsized":"2024-04-25T10:59:20.639078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:07.522883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.185571+0000","last_change":"2024-04-25T10:59:18.627083+0000","last_active":"2024-04-25T10:59:21.185571+0000","last_peered":"2024-04-25T10:59:21.185571+0000","last_clean":"2024-04-25T10:59:21.185571+0000","last_became_active":"2024-04-25T10:59:18.626450+0000","last_became_peered":"2024-04-25T10:59:18.626450+0000","last_unstale":"2024-04-25T10:59:21.185571+0000","last_undegraded":"2024-04-25T10:59:21.185571+0000","last_fullsized":"2024-04-25T10:59:21.185571+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:45.612221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228298+0000","last_change":"2024-04-25T10:59:19.653272+0000","last_active":"2024-04-25T10:59:21.228298+0000","last_peered":"2024-04-25T10:59:21.228298+0000","last_clean":"2024-04-25T10:59:21.228298+0000","last_became_active":"2024-04-25T10:59:19.652954+0000","last_became_peered":"2024-04-25T10:59:19.652954+0000","last_unstale":"2024-04-25T10:59:21.228298+0000","last_undegraded":"2024-04-25T10:59:21.228298+0000","last_fullsized":"2024-04-25T10:59:21.228298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:32.457404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638454+0000","last_change":"2024-04-25T10:59:18.630333+0000","last_active":"2024-04-25T10:59:20.638454+0000","last_peered":"2024-04-25T10:59:20.638454+0000","last_clean":"2024-04-25T10:59:20.638454+0000","last_became_active":"2024-04-25T10:59:18.629185+0000","last_became_peered":"2024-04-25T10:59:18.629185+0000","last_unstale":"2024-04-25T10:59:20.638454+0000","last_undegraded":"2024-04-25T10:59:20.638454+0000","last_fullsized":"2024-04-25T10:59:20.638454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:12.306399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228990+0000","last_change":"2024-04-25T10:59:19.654703+0000","last_active":"2024-04-25T10:59:21.228990+0000","last_peered":"2024-04-25T10:59:21.228990+0000","last_clean":"2024-04-25T10:59:21.228990+0000","last_became_active":"2024-04-25T10:59:19.654381+0000","last_became_peered":"2024-04-25T10:59:19.654381+0000","last_unstale":"2024-04-25T10:59:21.228990+0000","last_undegraded":"2024-04-25T10:59:21.228990+0000","last_fullsized":"2024-04-25T10:59:21.228990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:43.481266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.185751+0000","last_change":"2024-04-25T10:59:18.639591+0000","last_active":"2024-04-25T10:59:21.185751+0000","last_peered":"2024-04-25T10:59:21.185751+0000","last_clean":"2024-04-25T10:59:21.185751+0000","last_became_active":"2024-04-25T10:59:18.639276+0000","last_became_peered":"2024-04-25T10:59:18.639276+0000","last_unstale":"2024-04-25T10:59:21.185751+0000","last_undegraded":"2024-04-25T10:59:21.185751+0000","last_fullsized":"2024-04-25T10:59:21.185751+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:05.044315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.229162+0000","last_change":"2024-04-25T10:59:19.650496+0000","last_active":"2024-04-25T10:59:21.229162+0000","last_peered":"2024-04-25T10:59:21.229162+0000","last_clean":"2024-04-25T10:59:21.229162+0000","last_became_active":"2024-04-25T10:59:19.649725+0000","last_became_peered":"2024-04-25T10:59:19.649725+0000","last_unstale":"2024-04-25T10:59:21.229162+0000","last_undegraded":"2024-04-25T10:59:21.229162+0000","last_fullsized":"2024-04-25T10:59:21.229162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:53.860108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636578+0000","last_change":"2024-04-25T10:59:18.635647+0000","last_active":"2024-04-25T10:59:20.636578+0000","last_peered":"2024-04-25T10:59:20.636578+0000","last_clean":"2024-04-25T10:59:20.636578+0000","last_became_active":"2024-04-25T10:59:18.635264+0000","last_became_peered":"2024-04-25T10:59:18.635264+0000","last_unstale":"2024-04-25T10:59:20.636578+0000","last_undegraded":"2024-04-25T10:59:20.636578+0000","last_fullsized":"2024-04-25T10:59:20.636578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:45.612221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228718+0000","last_change":"2024-04-25T10:59:19.650869+0000","last_active":"2024-04-25T10:59:21.228718+0000","last_peered":"2024-04-25T10:59:21.228718+0000","last_clean":"2024-04-25T10:59:21.228718+0000","last_became_active":"2024-04-25T10:59:19.650541+0000","last_became_peered":"2024-04-25T10:59:19.650541+0000","last_unstale":"2024-04-25T10:59:21.228718+0000","last_undegraded":"2024-04-25T10:59:21.228718+0000","last_fullsized":"2024-04-25T10:59:21.228718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:14.189727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.185690+0000","last_change":"2024-04-25T10:59:18.637436+0000","last_active":"2024-04-25T10:59:21.185690+0000","last_peered":"2024-04-25T10:59:21.185690+0000","last_clean":"2024-04-25T10:59:21.185690+0000","last_became_active":"2024-04-25T10:59:18.637152+0000","last_became_peered":"2024-04-25T10:59:18.637152+0000","last_unstale":"2024-04-25T10:59:21.185690+0000","last_undegraded":"2024-04-25T10:59:21.185690+0000","last_fullsized":"2024-04-25T10:59:21.185690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:10.473815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228358+0000","last_change":"2024-04-25T10:59:19.655023+0000","last_active":"2024-04-25T10:59:21.228358+0000","last_peered":"2024-04-25T10:59:21.228358+0000","last_clean":"2024-04-25T10:59:21.228358+0000","last_became_active":"2024-04-25T10:59:19.654647+0000","last_became_peered":"2024-04-25T10:59:19.654647+0000","last_unstale":"2024-04-25T10:59:21.228358+0000","last_undegraded":"2024-04-25T10:59:21.228358+0000","last_fullsized":"2024-04-25T10:59:21.228358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:00.311029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":110,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227788+0000","last_change":"2024-04-25T10:59:07.839466+0000","last_active":"2024-04-25T10:59:21.227788+0000","last_peered":"2024-04-25T10:59:21.227788+0000","last_clean":"2024-04-25T10:59:21.227788+0000","last_became_active":"2024-04-25T10:59:07.836269+0000","last_became_peered":"2024-04-25T10:59:07.836269+0000","last_unstale":"2024-04-25T10:59:21.227788+0000","last_undegraded":"2024-04-25T10:59:21.227788+0000","last_fullsized":"2024-04-25T10:59:21.227788+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:06.549663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:06.549663+0000","last_clean_scrub_stamp":"2024-04-25T10:59:06.549663+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:21.575330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228265+0000","last_change":"2024-04-25T10:59:18.639570+0000","last_active":"2024-04-25T10:59:21.228265+0000","last_peered":"2024-04-25T10:59:21.228265+0000","last_clean":"2024-04-25T10:59:21.228265+0000","last_became_active":"2024-04-25T10:59:18.639231+0000","last_became_peered":"2024-04-25T10:59:18.639231+0000","last_unstale":"2024-04-25T10:59:21.228265+0000","last_undegraded":"2024-04-25T10:59:21.228265+0000","last_fullsized":"2024-04-25T10:59:21.228265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:28.594312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228305+0000","last_change":"2024-04-25T10:59:19.652991+0000","last_active":"2024-04-25T10:59:21.228305+0000","last_peered":"2024-04-25T10:59:21.228305+0000","last_clean":"2024-04-25T10:59:21.228305+0000","last_became_active":"2024-04-25T10:59:19.652680+0000","last_became_peered":"2024-04-25T10:59:19.652680+0000","last_unstale":"2024-04-25T10:59:21.228305+0000","last_undegraded":"2024-04-25T10:59:21.228305+0000","last_fullsized":"2024-04-25T10:59:21.228305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:38.297267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228139+0000","last_change":"2024-04-25T10:59:18.639571+0000","last_active":"2024-04-25T10:59:21.228139+0000","last_peered":"2024-04-25T10:59:21.228139+0000","last_clean":"2024-04-25T10:59:21.228139+0000","last_became_active":"2024-04-25T10:59:18.639065+0000","last_became_peered":"2024-04-25T10:59:18.639065+0000","last_unstale":"2024-04-25T10:59:21.228139+0000","last_undegraded":"2024-04-25T10:59:21.228139+0000","last_fullsized":"2024-04-25T10:59:21.228139+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:10.473815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227726+0000","last_change":"2024-04-25T10:59:19.644241+0000","last_active":"2024-04-25T10:59:21.227726+0000","last_peered":"2024-04-25T10:59:21.227726+0000","last_clean":"2024-04-25T10:59:21.227726+0000","last_became_active":"2024-04-25T10:59:19.643778+0000","last_became_peered":"2024-04-25T10:59:19.643778+0000","last_unstale":"2024-04-25T10:59:21.227726+0000","last_undegraded":"2024-04-25T10:59:21.227726+0000","last_fullsized":"2024-04-25T10:59:21.227726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:10.276726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.185642+0000","last_change":"2024-04-25T10:59:18.643811+0000","last_active":"2024-04-25T10:59:21.185642+0000","last_peered":"2024-04-25T10:59:21.185642+0000","last_clean":"2024-04-25T10:59:21.185642+0000","last_became_active":"2024-04-25T10:59:18.643454+0000","last_became_peered":"2024-04-25T10:59:18.643454+0000","last_unstale":"2024-04-25T10:59:21.185642+0000","last_undegraded":"2024-04-25T10:59:21.185642+0000","last_fullsized":"2024-04-25T10:59:21.185642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:09.933183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636785+0000","last_change":"2024-04-25T10:59:19.639259+0000","last_active":"2024-04-25T10:59:20.636785+0000","last_peered":"2024-04-25T10:59:20.636785+0000","last_clean":"2024-04-25T10:59:20.636785+0000","last_became_active":"2024-04-25T10:59:19.637680+0000","last_became_peered":"2024-04-25T10:59:19.637680+0000","last_unstale":"2024-04-25T10:59:20.636785+0000","last_undegraded":"2024-04-25T10:59:20.636785+0000","last_fullsized":"2024-04-25T10:59:20.636785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:33.050282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.229027+0000","last_change":"2024-04-25T10:59:18.643664+0000","last_active":"2024-04-25T10:59:21.229027+0000","last_peered":"2024-04-25T10:59:21.229027+0000","last_clean":"2024-04-25T10:59:21.229027+0000","last_became_active":"2024-04-25T10:59:18.643309+0000","last_became_peered":"2024-04-25T10:59:18.643309+0000","last_unstale":"2024-04-25T10:59:21.229027+0000","last_undegraded":"2024-04-25T10:59:21.229027+0000","last_fullsized":"2024-04-25T10:59:21.229027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:45.612221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.639084+0000","last_change":"2024-04-25T10:59:19.633507+0000","last_active":"2024-04-25T10:59:20.639084+0000","last_peered":"2024-04-25T10:59:20.639084+0000","last_clean":"2024-04-25T10:59:20.639084+0000","last_became_active":"2024-04-25T10:59:19.633182+0000","last_became_peered":"2024-04-25T10:59:19.633182+0000","last_unstale":"2024-04-25T10:59:20.639084+0000","last_undegraded":"2024-04-25T10:59:20.639084+0000","last_fullsized":"2024-04-25T10:59:20.639084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:11.481375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636672+0000","last_change":"2024-04-25T10:59:18.635772+0000","last_active":"2024-04-25T10:59:20.636672+0000","last_peered":"2024-04-25T10:59:20.636672+0000","last_clean":"2024-04-25T10:59:20.636672+0000","last_became_active":"2024-04-25T10:59:18.635486+0000","last_became_peered":"2024-04-25T10:59:18.635486+0000","last_unstale":"2024-04-25T10:59:20.636672+0000","last_undegraded":"2024-04-25T10:59:20.636672+0000","last_fullsized":"2024-04-25T10:59:20.636672+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:24.267022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228031+0000","last_change":"2024-04-25T10:59:19.648239+0000","last_active":"2024-04-25T10:59:21.228031+0000","last_peered":"2024-04-25T10:59:21.228031+0000","last_clean":"2024-04-25T10:59:21.228031+0000","last_became_active":"2024-04-25T10:59:19.646876+0000","last_became_peered":"2024-04-25T10:59:19.646876+0000","last_unstale":"2024-04-25T10:59:21.228031+0000","last_undegraded":"2024-04-25T10:59:21.228031+0000","last_fullsized":"2024-04-25T10:59:21.228031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:49.314956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228833+0000","last_change":"2024-04-25T10:59:18.643449+0000","last_active":"2024-04-25T10:59:21.228833+0000","last_peered":"2024-04-25T10:59:21.228833+0000","last_clean":"2024-04-25T10:59:21.228833+0000","last_became_active":"2024-04-25T10:59:18.643069+0000","last_became_peered":"2024-04-25T10:59:18.643069+0000","last_unstale":"2024-04-25T10:59:21.228833+0000","last_undegraded":"2024-04-25T10:59:21.228833+0000","last_fullsized":"2024-04-25T10:59:21.228833+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:57.176546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636600+0000","last_change":"2024-04-25T10:59:19.638854+0000","last_active":"2024-04-25T10:59:20.636600+0000","last_peered":"2024-04-25T10:59:20.636600+0000","last_clean":"2024-04-25T10:59:20.636600+0000","last_became_active":"2024-04-25T10:59:19.638273+0000","last_became_peered":"2024-04-25T10:59:19.638273+0000","last_unstale":"2024-04-25T10:59:20.636600+0000","last_undegraded":"2024-04-25T10:59:20.636600+0000","last_fullsized":"2024-04-25T10:59:20.636600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:15.132862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636693+0000","last_change":"2024-04-25T10:59:18.635651+0000","last_active":"2024-04-25T10:59:20.636693+0000","last_peered":"2024-04-25T10:59:20.636693+0000","last_clean":"2024-04-25T10:59:20.636693+0000","last_became_active":"2024-04-25T10:59:18.635348+0000","last_became_peered":"2024-04-25T10:59:18.635348+0000","last_unstale":"2024-04-25T10:59:20.636693+0000","last_undegraded":"2024-04-25T10:59:20.636693+0000","last_fullsized":"2024-04-25T10:59:20.636693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:58.505038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228269+0000","last_change":"2024-04-25T10:59:19.640663+0000","last_active":"2024-04-25T10:59:21.228269+0000","last_peered":"2024-04-25T10:59:21.228269+0000","last_clean":"2024-04-25T10:59:21.228269+0000","last_became_active":"2024-04-25T10:59:19.640311+0000","last_became_peered":"2024-04-25T10:59:19.640311+0000","last_unstale":"2024-04-25T10:59:21.228269+0000","last_undegraded":"2024-04-25T10:59:21.228269+0000","last_fullsized":"2024-04-25T10:59:21.228269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:57.344787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638534+0000","last_change":"2024-04-25T10:59:18.635706+0000","last_active":"2024-04-25T10:59:20.638534+0000","last_peered":"2024-04-25T10:59:20.638534+0000","last_clean":"2024-04-25T10:59:20.638534+0000","last_became_active":"2024-04-25T10:59:18.635377+0000","last_became_peered":"2024-04-25T10:59:18.635377+0000","last_unstale":"2024-04-25T10:59:20.638534+0000","last_undegraded":"2024-04-25T10:59:20.638534+0000","last_fullsized":"2024-04-25T10:59:20.638534+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:24.267022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227935+0000","last_change":"2024-04-25T10:59:19.635255+0000","last_active":"2024-04-25T10:59:21.227935+0000","last_peered":"2024-04-25T10:59:21.227935+0000","last_clean":"2024-04-25T10:59:21.227935+0000","last_became_active":"2024-04-25T10:59:19.634829+0000","last_became_peered":"2024-04-25T10:59:19.634829+0000","last_unstale":"2024-04-25T10:59:21.227935+0000","last_undegraded":"2024-04-25T10:59:21.227935+0000","last_fullsized":"2024-04-25T10:59:21.227935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:52.815596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227702+0000","last_change":"2024-04-25T10:59:18.643806+0000","last_active":"2024-04-25T10:59:21.227702+0000","last_peered":"2024-04-25T10:59:21.227702+0000","last_clean":"2024-04-25T10:59:21.227702+0000","last_became_active":"2024-04-25T10:59:18.643374+0000","last_became_peered":"2024-04-25T10:59:18.643374+0000","last_unstale":"2024-04-25T10:59:21.227702+0000","last_undegraded":"2024-04-25T10:59:21.227702+0000","last_fullsized":"2024-04-25T10:59:21.227702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:45.612221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.637981+0000","last_change":"2024-04-25T10:59:19.637663+0000","last_active":"2024-04-25T10:59:20.637981+0000","last_peered":"2024-04-25T10:59:20.637981+0000","last_clean":"2024-04-25T10:59:20.637981+0000","last_became_active":"2024-04-25T10:59:19.637309+0000","last_became_peered":"2024-04-25T10:59:19.637309+0000","last_unstale":"2024-04-25T10:59:20.637981+0000","last_undegraded":"2024-04-25T10:59:20.637981+0000","last_fullsized":"2024-04-25T10:59:20.637981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:29.601873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636484+0000","last_change":"2024-04-25T10:59:18.636340+0000","last_active":"2024-04-25T10:59:20.636484+0000","last_peered":"2024-04-25T10:59:20.636484+0000","last_clean":"2024-04-25T10:59:20.636484+0000","last_became_active":"2024-04-25T10:59:18.636036+0000","last_became_peered":"2024-04-25T10:59:18.636036+0000","last_unstale":"2024-04-25T10:59:20.636484+0000","last_undegraded":"2024-04-25T10:59:20.636484+0000","last_fullsized":"2024-04-25T10:59:20.636484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:15.754476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227666+0000","last_change":"2024-04-25T10:59:19.632935+0000","last_active":"2024-04-25T10:59:21.227666+0000","last_peered":"2024-04-25T10:59:21.227666+0000","last_clean":"2024-04-25T10:59:21.227666+0000","last_became_active":"2024-04-25T10:59:19.632658+0000","last_became_peered":"2024-04-25T10:59:19.632658+0000","last_unstale":"2024-04-25T10:59:21.227666+0000","last_undegraded":"2024-04-25T10:59:21.227666+0000","last_fullsized":"2024-04-25T10:59:21.227666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:52.815596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636560+0000","last_change":"2024-04-25T10:59:18.639304+0000","last_active":"2024-04-25T10:59:20.636560+0000","last_peered":"2024-04-25T10:59:20.636560+0000","last_clean":"2024-04-25T10:59:20.636560+0000","last_became_active":"2024-04-25T10:59:18.638939+0000","last_became_peered":"2024-04-25T10:59:18.638939+0000","last_unstale":"2024-04-25T10:59:20.636560+0000","last_undegraded":"2024-04-25T10:59:20.636560+0000","last_fullsized":"2024-04-25T10:59:20.636560+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:28.594312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227866+0000","last_change":"2024-04-25T10:59:19.631658+0000","last_active":"2024-04-25T10:59:21.227866+0000","last_peered":"2024-04-25T10:59:21.227866+0000","last_clean":"2024-04-25T10:59:21.227866+0000","last_became_active":"2024-04-25T10:59:19.631363+0000","last_became_peered":"2024-04-25T10:59:19.631363+0000","last_unstale":"2024-04-25T10:59:21.227866+0000","last_undegraded":"2024-04-25T10:59:21.227866+0000","last_fullsized":"2024-04-25T10:59:21.227866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:33.050282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228855+0000","last_change":"2024-04-25T10:59:18.638514+0000","last_active":"2024-04-25T10:59:21.228855+0000","last_peered":"2024-04-25T10:59:21.228855+0000","last_clean":"2024-04-25T10:59:21.228855+0000","last_became_active":"2024-04-25T10:59:18.638113+0000","last_became_peered":"2024-04-25T10:59:18.638113+0000","last_unstale":"2024-04-25T10:59:21.228855+0000","last_undegraded":"2024-04-25T10:59:21.228855+0000","last_fullsized":"2024-04-25T10:59:21.228855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:09.933183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638752+0000","last_change":"2024-04-25T10:59:19.652586+0000","last_active":"2024-04-25T10:59:20.638752+0000","last_peered":"2024-04-25T10:59:20.638752+0000","last_clean":"2024-04-25T10:59:20.638752+0000","last_became_active":"2024-04-25T10:59:19.652246+0000","last_became_peered":"2024-04-25T10:59:19.652246+0000","last_unstale":"2024-04-25T10:59:20.638752+0000","last_undegraded":"2024-04-25T10:59:20.638752+0000","last_fullsized":"2024-04-25T10:59:20.638752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:52.815596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228042+0000","last_change":"2024-04-25T10:59:19.649474+0000","last_active":"2024-04-25T10:59:21.228042+0000","last_peered":"2024-04-25T10:59:21.228042+0000","last_clean":"2024-04-25T10:59:21.228042+0000","last_became_active":"2024-04-25T10:59:19.649134+0000","last_became_peered":"2024-04-25T10:59:19.649134+0000","last_unstale":"2024-04-25T10:59:21.228042+0000","last_undegraded":"2024-04-25T10:59:21.228042+0000","last_fullsized":"2024-04-25T10:59:21.228042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:38.297267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228094+0000","last_change":"2024-04-25T10:59:18.641120+0000","last_active":"2024-04-25T10:59:21.228094+0000","last_peered":"2024-04-25T10:59:21.228094+0000","last_clean":"2024-04-25T10:59:21.228094+0000","last_became_active":"2024-04-25T10:59:18.640791+0000","last_became_peered":"2024-04-25T10:59:18.640791+0000","last_unstale":"2024-04-25T10:59:21.228094+0000","last_undegraded":"2024-04-25T10:59:21.228094+0000","last_fullsized":"2024-04-25T10:59:21.228094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:43.470493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228726+0000","last_change":"2024-04-25T10:59:19.654395+0000","last_active":"2024-04-25T10:59:21.228726+0000","last_peered":"2024-04-25T10:59:21.228726+0000","last_clean":"2024-04-25T10:59:21.228726+0000","last_became_active":"2024-04-25T10:59:19.654107+0000","last_became_peered":"2024-04-25T10:59:19.654107+0000","last_unstale":"2024-04-25T10:59:21.228726+0000","last_undegraded":"2024-04-25T10:59:21.228726+0000","last_fullsized":"2024-04-25T10:59:21.228726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:33.154184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228501+0000","last_change":"2024-04-25T10:59:18.640843+0000","last_active":"2024-04-25T10:59:21.228501+0000","last_peered":"2024-04-25T10:59:21.228501+0000","last_clean":"2024-04-25T10:59:21.228501+0000","last_became_active":"2024-04-25T10:59:18.640496+0000","last_became_peered":"2024-04-25T10:59:18.640496+0000","last_unstale":"2024-04-25T10:59:21.228501+0000","last_undegraded":"2024-04-25T10:59:21.228501+0000","last_fullsized":"2024-04-25T10:59:21.228501+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:05.044315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227754+0000","last_change":"2024-04-25T10:59:19.652337+0000","last_active":"2024-04-25T10:59:21.227754+0000","last_peered":"2024-04-25T10:59:21.227754+0000","last_clean":"2024-04-25T10:59:21.227754+0000","last_became_active":"2024-04-25T10:59:19.652008+0000","last_became_peered":"2024-04-25T10:59:19.652008+0000","last_unstale":"2024-04-25T10:59:21.227754+0000","last_undegraded":"2024-04-25T10:59:21.227754+0000","last_fullsized":"2024-04-25T10:59:21.227754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:32:26.553467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228556+0000","last_change":"2024-04-25T10:59:18.643508+0000","last_active":"2024-04-25T10:59:21.228556+0000","last_peered":"2024-04-25T10:59:21.228556+0000","last_clean":"2024-04-25T10:59:21.228556+0000","last_became_active":"2024-04-25T10:59:18.643186+0000","last_became_peered":"2024-04-25T10:59:18.643186+0000","last_unstale":"2024-04-25T10:59:21.228556+0000","last_undegraded":"2024-04-25T10:59:21.228556+0000","last_fullsized":"2024-04-25T10:59:21.228556+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:35.341389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228500+0000","last_change":"2024-04-25T10:59:19.640778+0000","last_active":"2024-04-25T10:59:21.228500+0000","last_peered":"2024-04-25T10:59:21.228500+0000","last_clean":"2024-04-25T10:59:21.228500+0000","last_became_active":"2024-04-25T10:59:19.640479+0000","last_became_peered":"2024-04-25T10:59:19.640479+0000","last_unstale":"2024-04-25T10:59:21.228500+0000","last_undegraded":"2024-04-25T10:59:21.228500+0000","last_fullsized":"2024-04-25T10:59:21.228500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:52.334437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228246+0000","last_change":"2024-04-25T10:59:18.635569+0000","last_active":"2024-04-25T10:59:21.228246+0000","last_peered":"2024-04-25T10:59:21.228246+0000","last_clean":"2024-04-25T10:59:21.228246+0000","last_became_active":"2024-04-25T10:59:18.635303+0000","last_became_peered":"2024-04-25T10:59:18.635303+0000","last_unstale":"2024-04-25T10:59:21.228246+0000","last_undegraded":"2024-04-25T10:59:21.228246+0000","last_fullsized":"2024-04-25T10:59:21.228246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:46.279987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":790528,"data_stored":655584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":196608,"data_stored":64724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":66246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":798720,"data_stored":655640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:59:22.981 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:59:22.981 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T10:59:22.981 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T10:59:22.981 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T10:59:22.982 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T10:59:23.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.124+0000 7f114872d640 1 Processor -- start 2024-04-25T10:59:23.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.124+0000 7f114872d640 1 -- start start 2024-04-25T10:59:23.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.125+0000 7f114872d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1140160870 0x7f11400b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.125+0000 7f114872d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f11400d2410 0x7f11400b2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.125+0000 7f114872d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11400cfb30 con 0x7f11400cf300 2024-04-25T10:59:23.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.125+0000 7f114872d640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f11400d00f0 con 0x7f11400d2410 2024-04-25T10:59:23.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.125+0000 7f114872d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f11400b2470 con 0x7f1140160870 2024-04-25T10:59:23.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.126+0000 7f11464c9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1140160870 0x7f11400b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.126+0000 7f1146cca640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f11400d2410 0x7f11400b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.126+0000 7f11464c9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1140160870 0x7f11400b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.69:58086/0 (socket says 172.21.15.69:58086) 2024-04-25T10:59:23.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.126+0000 7f11464c9640 1 -- 172.21.15.69:0/1479462657 learned_addr learned my addr 172.21.15.69:0/1479462657 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:23.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.126+0000 7f11464c9640 1 -- 172.21.15.69:0/1479462657 >> v2:172.21.15.146:3300/0 conn(0x7f11400d2410 msgr2=0x7f11400b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:23.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.126+0000 7f11464c9640 1 --2- 172.21.15.69:0/1479462657 >> v2:172.21.15.146:3300/0 conn(0x7f11400d2410 0x7f11400b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.126+0000 7f11464c9640 1 -- 172.21.15.69:0/1479462657 >> v1:172.21.15.146:6789/0 conn(0x7f11400cf300 legacy=0x7f11400d1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.126+0000 7f11464c9640 1 -- 172.21.15.69:0/1479462657 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11400cfe10 con 0x7f1140160870 2024-04-25T10:59:23.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.127+0000 7f11464c9640 1 --2- 172.21.15.69:0/1479462657 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1140160870 0x7f11400b3f80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f113c009870 tx=0x7f113c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=673edbcc02a69f1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:23.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.127+0000 7f11454c7640 1 -- 172.21.15.69:0/1479462657 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c037070 con 0x7f1140160870 2024-04-25T10:59:23.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.127+0000 7f11454c7640 1 -- 172.21.15.69:0/1479462657 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f113c032440 con 0x7f1140160870 2024-04-25T10:59:23.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.127+0000 7f11454c7640 1 -- 172.21.15.69:0/1479462657 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c032810 con 0x7f1140160870 2024-04-25T10:59:23.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.127+0000 7f114872d640 1 -- 172.21.15.69:0/1479462657 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1140160870 msgr2=0x7f11400b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.127+0000 7f114872d640 1 --2- 172.21.15.69:0/1479462657 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1140160870 0x7f11400b3f80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f113c009870 tx=0x7f113c02a270 comp rx=0 tx=0).stop 2024-04-25T10:59:23.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.128+0000 7f114872d640 1 -- 172.21.15.69:0/1479462657 shutdown_connections 2024-04-25T10:59:23.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.128+0000 7f114872d640 1 --2- 172.21.15.69:0/1479462657 >> v2:172.21.15.146:3300/0 conn(0x7f11400d2410 0x7f11400b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.128+0000 7f114872d640 1 --2- 172.21.15.69:0/1479462657 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1140160870 0x7f11400b3f80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.128+0000 7f114872d640 1 -- 172.21.15.69:0/1479462657 >> 172.21.15.69:0/1479462657 conn(0x7f114009b7d0 msgr2=0x7f114009bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:23.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.128+0000 7f114872d640 1 -- 172.21.15.69:0/1479462657 shutdown_connections 2024-04-25T10:59:23.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.128+0000 7f114872d640 1 -- 172.21.15.69:0/1479462657 wait complete. 2024-04-25T10:59:23.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.129+0000 7f114872d640 1 Processor -- start 2024-04-25T10:59:23.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.129+0000 7f114872d640 1 -- start start 2024-04-25T10:59:23.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.131+0000 7f114872d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11400cf300 0x7f11401fab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.131+0000 7f114872d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11400d2410 0x7f114015b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.131+0000 7f114872d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1140160870 0x7f11400b4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.131+0000 7f114872d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f11400af9d0 con 0x7f1140160870 2024-04-25T10:59:23.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.131+0000 7f114872d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f11400af850 con 0x7f11400cf300 2024-04-25T10:59:23.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.131+0000 7f114872d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f11400cf710 con 0x7f11400d2410 2024-04-25T10:59:23.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.131+0000 7f1146cca640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1140160870 0x7f11400b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.131+0000 7f1146cca640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1140160870 0x7f11400b4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41202/0 (socket says 172.21.15.69:41202) 2024-04-25T10:59:23.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.131+0000 7f1146cca640 1 -- 172.21.15.69:0/2280523938 learned_addr learned my addr 172.21.15.69:0/2280523938 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:23.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.131+0000 7f11464c9640 1 --2- 172.21.15.69:0/2280523938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11400cf300 0x7f11401fab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.131+0000 7f1145cc8640 1 --2- 172.21.15.69:0/2280523938 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11400d2410 0x7f114015b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.131+0000 7f1146cca640 1 -- 172.21.15.69:0/2280523938 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11400d2410 msgr2=0x7f114015b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.131+0000 7f1146cca640 1 --2- 172.21.15.69:0/2280523938 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11400d2410 0x7f114015b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.131+0000 7f1146cca640 1 -- 172.21.15.69:0/2280523938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11400cf300 msgr2=0x7f11401fab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.131+0000 7f1146cca640 1 --2- 172.21.15.69:0/2280523938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11400cf300 0x7f11401fab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.131+0000 7f1146cca640 1 -- 172.21.15.69:0/2280523938 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114015b560 con 0x7f1140160870 2024-04-25T10:59:23.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.131+0000 7f1146cca640 1 --2- 172.21.15.69:0/2280523938 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1140160870 0x7f11400b4530 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f113800efc0 tx=0x7f113800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:23.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.131+0000 7f1136ffd640 1 -- 172.21.15.69:0/2280523938 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1138018020 con 0x7f1140160870 2024-04-25T10:59:23.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.131+0000 7f1136ffd640 1 -- 172.21.15.69:0/2280523938 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1138014480 con 0x7f1140160870 2024-04-25T10:59:23.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.131+0000 7f1136ffd640 1 -- 172.21.15.69:0/2280523938 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1138014870 con 0x7f1140160870 2024-04-25T10:59:23.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.131+0000 7f114872d640 1 -- 172.21.15.69:0/2280523938 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11401631b0 con 0x7f1140160870 2024-04-25T10:59:23.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.132+0000 7f114872d640 1 -- 172.21.15.69:0/2280523938 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1140163790 con 0x7f1140160870 2024-04-25T10:59:23.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.133+0000 7f114872d640 1 -- 172.21.15.69:0/2280523938 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11400b7760 con 0x7f1140160870 2024-04-25T10:59:23.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.133+0000 7f1136ffd640 1 -- 172.21.15.69:0/2280523938 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1138005d10 con 0x7f1140160870 2024-04-25T10:59:23.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.133+0000 7f1136ffd640 1 --2- 172.21.15.69:0/2280523938 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f11240744e0 0x7f11240769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.134+0000 7f1136ffd640 1 -- 172.21.15.69:0/2280523938 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1138095e30 con 0x7f1140160870 2024-04-25T10:59:23.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.134+0000 7f11464c9640 1 --2- 172.21.15.69:0/2280523938 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f11240744e0 0x7f11240769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.134+0000 7f11464c9640 1 --2- 172.21.15.69:0/2280523938 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f11240744e0 0x7f11240769f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f113c034000 tx=0x7f113c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:23.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.138+0000 7f1136ffd640 1 -- 172.21.15.69:0/2280523938 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f11400b7760 con 0x7f1140160870 2024-04-25T10:59:23.235 INFO:tasks.ceph.mds.b.smithi146.stderr:2024-04-25T10:59:23.234+0000 7faf039d1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:59:23.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.360+0000 7f114872d640 1 -- 172.21.15.69:0/2280523938 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f1140050740 con 0x7f1140160870 2024-04-25T10:59:23.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.363+0000 7f1136ffd640 1 -- 172.21.15.69:0/2280523938 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+186741 (secure 0 0 0) 0x7f1138022030 con 0x7f1140160870 2024-04-25T10:59:23.366 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:23.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.366+0000 7f114872d640 1 -- 172.21.15.69:0/2280523938 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f11240744e0 msgr2=0x7f11240769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.366+0000 7f114872d640 1 --2- 172.21.15.69:0/2280523938 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f11240744e0 0x7f11240769f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f113c034000 tx=0x7f113c02e000 comp rx=0 tx=0).stop 2024-04-25T10:59:23.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.366+0000 7f114872d640 1 -- 172.21.15.69:0/2280523938 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1140160870 msgr2=0x7f11400b4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.366+0000 7f114872d640 1 --2- 172.21.15.69:0/2280523938 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1140160870 0x7f11400b4530 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f113800efc0 tx=0x7f113800c490 comp rx=0 tx=0).stop 2024-04-25T10:59:23.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.366+0000 7f114872d640 1 -- 172.21.15.69:0/2280523938 shutdown_connections 2024-04-25T10:59:23.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.366+0000 7f114872d640 1 --2- 172.21.15.69:0/2280523938 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f11240744e0 0x7f11240769f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.366+0000 7f114872d640 1 --2- 172.21.15.69:0/2280523938 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1140160870 0x7f11400b4530 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.366+0000 7f114872d640 1 --2- 172.21.15.69:0/2280523938 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11400d2410 0x7f114015b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.366+0000 7f114872d640 1 --2- 172.21.15.69:0/2280523938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11400cf300 0x7f11401fab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.366+0000 7f114872d640 1 -- 172.21.15.69:0/2280523938 >> 172.21.15.69:0/2280523938 conn(0x7f114009b7d0 msgr2=0x7f11400707e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:23.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.366+0000 7f114872d640 1 -- 172.21.15.69:0/2280523938 shutdown_connections 2024-04-25T10:59:23.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.366+0000 7f114872d640 1 -- 172.21.15.69:0/2280523938 wait complete. 2024-04-25T10:59:23.378 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":5,"flags":0,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":1107421267},{"type":"v1","addr":"172.21.15.69:6833","nonce":1107421267}]},"active_addr":"172.21.15.69:6833/1107421267","active_change":"2024-04-25T10:58:58.394057+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4100,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":476530032}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":3647014514}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":222532613}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":1381182824}]}]} 2024-04-25T10:59:23.381 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T10:59:23.381 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T10:59:23.381 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:59:23.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.488+0000 7ff16236c640 1 Processor -- start 2024-04-25T10:59:23.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.489+0000 7ff16236c640 1 -- start start 2024-04-25T10:59:23.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.490+0000 7ff16236c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff15c1646b0 0x7ff15c164ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.490+0000 7ff16236c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff15c0ce060 0x7ff15c0ce470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.490+0000 7ff16236c640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff15c06ed60 con 0x7ff15c0b8ca0 2024-04-25T10:59:23.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.490+0000 7ff16236c640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7ff15c0d35a0 con 0x7ff15c1646b0 2024-04-25T10:59:23.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.490+0000 7ff16236c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff15c06e8d0 con 0x7ff15c0ce060 2024-04-25T10:59:23.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.490+0000 7ff15bfff640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7ff15c0b8ca0 0x7ff15c163fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:57152/0 (socket says 172.21.15.69:57152) 2024-04-25T10:59:23.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.490+0000 7ff15bfff640 1 -- 172.21.15.69:0/906772682 learned_addr learned my addr 172.21.15.69:0/906772682 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:23.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.490+0000 7ff15b7fe640 1 --2- 172.21.15.69:0/906772682 >> v2:172.21.15.69:3300/0 conn(0x7ff15c1646b0 0x7ff15c164ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.490+0000 7ff160909640 1 --2- 172.21.15.69:0/906772682 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff15c0ce060 0x7ff15c0ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.490+0000 7ff15b7fe640 1 -- 172.21.15.69:0/906772682 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff15c0ce060 msgr2=0x7ff15c0ce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:23.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.490+0000 7ff15b7fe640 1 --2- 172.21.15.69:0/906772682 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff15c0ce060 0x7ff15c0ce470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.490+0000 7ff15b7fe640 1 -- 172.21.15.69:0/906772682 >> v1:172.21.15.69:6789/0 conn(0x7ff15c0b8ca0 legacy=0x7ff15c163fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.490+0000 7ff15affd640 1 -- 172.21.15.69:0/906772682 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 9891031 0 0) 0x7ff15c06ed60 con 0x7ff15c0b8ca0 2024-04-25T10:59:23.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.490+0000 7ff15b7fe640 1 -- 172.21.15.69:0/906772682 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff15c1608b0 con 0x7ff15c1646b0 2024-04-25T10:59:23.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.490+0000 7ff160909640 1 --2- 172.21.15.69:0/906772682 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff15c0ce060 0x7ff15c0ce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:23.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.491+0000 7ff15b7fe640 1 --2- 172.21.15.69:0/906772682 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff15c1646b0 0x7ff15c164ac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff1500097a0 tx=0x7ff15002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc68116e575543 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:23.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.491+0000 7ff15affd640 1 -- 172.21.15.69:0/906772682 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff150037070 con 0x7ff15c1646b0 2024-04-25T10:59:23.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.491+0000 7ff15affd640 1 -- 172.21.15.69:0/906772682 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff150032440 con 0x7ff15c1646b0 2024-04-25T10:59:23.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.491+0000 7ff15affd640 1 -- 172.21.15.69:0/906772682 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff150032810 con 0x7ff15c1646b0 2024-04-25T10:59:23.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.492+0000 7ff16236c640 1 -- 172.21.15.69:0/906772682 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff15c1646b0 msgr2=0x7ff15c164ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.492+0000 7ff16236c640 1 --2- 172.21.15.69:0/906772682 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff15c1646b0 0x7ff15c164ac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff1500097a0 tx=0x7ff15002a180 comp rx=0 tx=0).stop 2024-04-25T10:59:23.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.492+0000 7ff16236c640 1 -- 172.21.15.69:0/906772682 shutdown_connections 2024-04-25T10:59:23.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.492+0000 7ff16236c640 1 --2- 172.21.15.69:0/906772682 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff15c0ce060 0x7ff15c0ce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.492+0000 7ff16236c640 1 --2- 172.21.15.69:0/906772682 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff15c1646b0 0x7ff15c164ac0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.492+0000 7ff16236c640 1 -- 172.21.15.69:0/906772682 >> 172.21.15.69:0/906772682 conn(0x7ff15c0d3aa0 msgr2=0x7ff15c0d3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:23.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.492+0000 7ff16236c640 1 -- 172.21.15.69:0/906772682 shutdown_connections 2024-04-25T10:59:23.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.492+0000 7ff16236c640 1 -- 172.21.15.69:0/906772682 wait complete. 2024-04-25T10:59:23.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.494+0000 7ff16236c640 1 Processor -- start 2024-04-25T10:59:23.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.494+0000 7ff16236c640 1 -- start start 2024-04-25T10:59:23.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.495+0000 7ff16236c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff15c0b8ca0 0x7ff15c0cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.495+0000 7ff16236c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff15c0ce060 0x7ff15c158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.495+0000 7ff16236c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff15c1646b0 0x7ff15c162a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.495+0000 7ff16236c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff15c06e560 con 0x7ff15c1646b0 2024-04-25T10:59:23.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.495+0000 7ff16236c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff15c06e280 con 0x7ff15c0ce060 2024-04-25T10:59:23.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.495+0000 7ff16236c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff15c0d2f40 con 0x7ff15c0b8ca0 2024-04-25T10:59:23.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.495+0000 7ff160909640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff15c1646b0 0x7ff15c162a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.495+0000 7ff160909640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff15c1646b0 0x7ff15c162a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41216/0 (socket says 172.21.15.69:41216) 2024-04-25T10:59:23.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.495+0000 7ff160909640 1 -- 172.21.15.69:0/3335815199 learned_addr learned my addr 172.21.15.69:0/3335815199 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:23.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.496+0000 7ff15b7fe640 1 --2- 172.21.15.69:0/3335815199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff15c0ce060 0x7ff15c158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.496+0000 7ff15bfff640 1 --2- 172.21.15.69:0/3335815199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff15c0b8ca0 0x7ff15c0cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.496+0000 7ff160909640 1 -- 172.21.15.69:0/3335815199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff15c0b8ca0 msgr2=0x7ff15c0cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.496+0000 7ff160909640 1 --2- 172.21.15.69:0/3335815199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff15c0b8ca0 0x7ff15c0cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.496+0000 7ff160909640 1 -- 172.21.15.69:0/3335815199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff15c0ce060 msgr2=0x7ff15c158310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.496+0000 7ff160909640 1 --2- 172.21.15.69:0/3335815199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff15c0ce060 0x7ff15c158310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.496+0000 7ff160909640 1 -- 172.21.15.69:0/3335815199 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff15c163740 con 0x7ff15c1646b0 2024-04-25T10:59:23.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.496+0000 7ff160909640 1 --2- 172.21.15.69:0/3335815199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff15c1646b0 0x7ff15c162a10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff15400cdb0 tx=0x7ff15400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:23.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.496+0000 7ff158ff9640 1 -- 172.21.15.69:0/3335815199 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15400f680 con 0x7ff15c1646b0 2024-04-25T10:59:23.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.496+0000 7ff158ff9640 1 -- 172.21.15.69:0/3335815199 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff15400f840 con 0x7ff15c1646b0 2024-04-25T10:59:23.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.496+0000 7ff158ff9640 1 -- 172.21.15.69:0/3335815199 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15400fc30 con 0x7ff15c1646b0 2024-04-25T10:59:23.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.496+0000 7ff16236c640 1 -- 172.21.15.69:0/3335815199 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff15c0cacf0 con 0x7ff15c1646b0 2024-04-25T10:59:23.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.496+0000 7ff16236c640 1 -- 172.21.15.69:0/3335815199 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff15c0cb2b0 con 0x7ff15c1646b0 2024-04-25T10:59:23.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.497+0000 7ff16236c640 1 -- 172.21.15.69:0/3335815199 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff15c0caae0 con 0x7ff15c1646b0 2024-04-25T10:59:23.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.498+0000 7ff158ff9640 1 -- 172.21.15.69:0/3335815199 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff1540213f0 con 0x7ff15c1646b0 2024-04-25T10:59:23.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.498+0000 7ff158ff9640 1 --2- 172.21.15.69:0/3335815199 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff1480744b0 0x7ff1480769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.498+0000 7ff15bfff640 1 --2- 172.21.15.69:0/3335815199 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff1480744b0 0x7ff1480769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.498+0000 7ff158ff9640 1 -- 172.21.15.69:0/3335815199 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff154095f90 con 0x7ff15c1646b0 2024-04-25T10:59:23.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.499+0000 7ff15bfff640 1 --2- 172.21.15.69:0/3335815199 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff1480744b0 0x7ff1480769c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff14c000f60 tx=0x7ff14c007c00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:23.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.503+0000 7ff158ff9640 1 -- 172.21.15.69:0/3335815199 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff154014030 con 0x7ff15c1646b0 2024-04-25T10:59:23.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.679+0000 7ff16236c640 1 -- 172.21.15.69:0/3335815199 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff15c0d4700 con 0x7ff15c1646b0 2024-04-25T10:59:23.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.680+0000 7ff158ff9640 1 -- 172.21.15.69:0/3335815199 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7ff1540607b0 con 0x7ff15c1646b0 2024-04-25T10:59:23.681 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:23.681 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":20,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T10:59:20.630151+0000","last_up_change":"2024-04-25T10:59:05.546839+0000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:59:06.422674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:59:16.727980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:59:17.938486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6825","nonce":1876101065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6827","nonce":1876101065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6831","nonce":1876101065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6829","nonce":1876101065}]},"public_addr":"172.21.15.69:6825/1876101065","cluster_addr":"172.21.15.69:6827/1876101065","heartbeat_back_addr":"172.21.15.69:6831/1876101065","heartbeat_front_addr":"172.21.15.69:6829/1876101065","state":["exists","up"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6801","nonce":1806438563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6803","nonce":1806438563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6807","nonce":1806438563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6805","nonce":1806438563}]},"public_addr":"172.21.15.69:6801/1806438563","cluster_addr":"172.21.15.69:6803/1806438563","heartbeat_back_addr":"172.21.15.69:6807/1806438563","heartbeat_front_addr":"172.21.15.69:6805/1806438563","state":["exists","up"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6817","nonce":1593617983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6819","nonce":1593617983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6823","nonce":1593617983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6821","nonce":1593617983}]},"public_addr":"172.21.15.69:6817/1593617983","cluster_addr":"172.21.15.69:6819/1593617983","heartbeat_back_addr":"172.21.15.69:6823/1593617983","heartbeat_front_addr":"172.21.15.69:6821/1593617983","state":["exists","up"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6809","nonce":3950661294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6811","nonce":3950661294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6815","nonce":3950661294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6813","nonce":3950661294}]},"public_addr":"172.21.15.69:6809/3950661294","cluster_addr":"172.21.15.69:6811/3950661294","heartbeat_back_addr":"172.21.15.69:6815/3950661294","heartbeat_front_addr":"172.21.15.69:6813/3950661294","state":["exists","up"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6801","nonce":2508929872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6803","nonce":2508929872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6807","nonce":2508929872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6805","nonce":2508929872}]},"public_addr":"172.21.15.146:6801/2508929872","cluster_addr":"172.21.15.146:6803/2508929872","heartbeat_back_addr":"172.21.15.146:6807/2508929872","heartbeat_front_addr":"172.21.15.146:6805/2508929872","state":["exists","up"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6811","nonce":1310911138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6816","nonce":1310911138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6823","nonce":1310911138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6821","nonce":1310911138}]},"public_addr":"172.21.15.146:6811/1310911138","cluster_addr":"172.21.15.146:6816/1310911138","heartbeat_back_addr":"172.21.15.146:6823/1310911138","heartbeat_front_addr":"172.21.15.146:6821/1310911138","state":["exists","up"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6810","nonce":634688021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6813","nonce":634688021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6820","nonce":634688021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6817","nonce":634688021}]},"public_addr":"172.21.15.146:6810/634688021","cluster_addr":"172.21.15.146:6813/634688021","heartbeat_back_addr":"172.21.15.146:6820/634688021","heartbeat_front_addr":"172.21.15.146:6817/634688021","state":["exists","up"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6825","nonce":3814215935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6827","nonce":3814215935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6831","nonce":3814215935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6829","nonce":3814215935}]},"public_addr":"172.21.15.146:6825/3814215935","cluster_addr":"172.21.15.146:6827/3814215935","heartbeat_back_addr":"172.21.15.146:6831/3814215935","heartbeat_front_addr":"172.21.15.146:6829/3814215935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.820200+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.850189+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.764560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.866248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.807897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.869881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.782310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.827131+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:59:23.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.682+0000 7ff16236c640 1 -- 172.21.15.69:0/3335815199 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff1480744b0 msgr2=0x7ff1480769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.682+0000 7ff16236c640 1 --2- 172.21.15.69:0/3335815199 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff1480744b0 0x7ff1480769c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff14c000f60 tx=0x7ff14c007c00 comp rx=0 tx=0).stop 2024-04-25T10:59:23.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.682+0000 7ff16236c640 1 -- 172.21.15.69:0/3335815199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff15c1646b0 msgr2=0x7ff15c162a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.682+0000 7ff16236c640 1 --2- 172.21.15.69:0/3335815199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff15c1646b0 0x7ff15c162a10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff15400cdb0 tx=0x7ff15400a650 comp rx=0 tx=0).stop 2024-04-25T10:59:23.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.682+0000 7ff16236c640 1 -- 172.21.15.69:0/3335815199 shutdown_connections 2024-04-25T10:59:23.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.682+0000 7ff16236c640 1 --2- 172.21.15.69:0/3335815199 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff1480744b0 0x7ff1480769c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.682+0000 7ff16236c640 1 --2- 172.21.15.69:0/3335815199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff15c1646b0 0x7ff15c162a10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.682+0000 7ff16236c640 1 --2- 172.21.15.69:0/3335815199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff15c0ce060 0x7ff15c158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.682+0000 7ff16236c640 1 --2- 172.21.15.69:0/3335815199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff15c0b8ca0 0x7ff15c0cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.682+0000 7ff16236c640 1 -- 172.21.15.69:0/3335815199 >> 172.21.15.69:0/3335815199 conn(0x7ff15c0d3aa0 msgr2=0x7ff15c0d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:23.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.682+0000 7ff16236c640 1 -- 172.21.15.69:0/3335815199 shutdown_connections 2024-04-25T10:59:23.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.682+0000 7ff16236c640 1 -- 172.21.15.69:0/3335815199 wait complete. 2024-04-25T10:59:23.695 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T10:59:23.695 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:59:23.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.801+0000 7f29962f2640 1 Processor -- start 2024-04-25T10:59:23.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.801+0000 7f29962f2640 1 -- start start 2024-04-25T10:59:23.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.802+0000 7f29962f2640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f29900b8ca0 0x7f2990163fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.802+0000 7f29962f2640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f29901644e0 0x7f29901648f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.802+0000 7f29962f2640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f29900d1f30 0x7f29900d2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.802+0000 7f29962f2640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f29900d35a0 con 0x7f29900b8ca0 2024-04-25T10:59:23.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.802+0000 7f29962f2640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f299006e8d0 con 0x7f29900d1f30 2024-04-25T10:59:23.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.802+0000 7f29962f2640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f299006ed60 con 0x7f29901644e0 2024-04-25T10:59:23.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.802+0000 7f298ffff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f29900b8ca0 0x7f2990163fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.802+0000 7f298ffff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f29900b8ca0 0x7f2990163fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41230/0 (socket says 172.21.15.69:41230) 2024-04-25T10:59:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.802+0000 7f298ffff640 1 -- 172.21.15.69:0/857115728 learned_addr learned my addr 172.21.15.69:0/857115728 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.803+0000 7f299488f640 1 --2- 172.21.15.69:0/857115728 >> v2:172.21.15.146:3300/0 conn(0x7f29900d1f30 0x7f29900d2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.803+0000 7f298f7fe640 1 --2- 172.21.15.69:0/857115728 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f29901644e0 0x7f29901648f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.803+0000 7f298ffff640 1 -- 172.21.15.69:0/857115728 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f29901644e0 msgr2=0x7f29901648f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.803+0000 7f298ffff640 1 --2- 172.21.15.69:0/857115728 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f29901644e0 0x7f29901648f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.803+0000 7f298ffff640 1 -- 172.21.15.69:0/857115728 >> v2:172.21.15.146:3300/0 conn(0x7f29900d1f30 msgr2=0x7f29900d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.803+0000 7f298ffff640 1 --2- 172.21.15.69:0/857115728 >> v2:172.21.15.146:3300/0 conn(0x7f29900d1f30 0x7f29900d2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.803+0000 7f298ffff640 1 -- 172.21.15.69:0/857115728 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29901608b0 con 0x7f29900b8ca0 2024-04-25T10:59:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.803+0000 7f298f7fe640 1 --2- 172.21.15.69:0/857115728 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f29901644e0 0x7f29901648f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.803+0000 7f299488f640 1 --2- 172.21.15.69:0/857115728 >> v2:172.21.15.146:3300/0 conn(0x7f29900d1f30 0x7f29900d2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.803+0000 7f298ffff640 1 --2- 172.21.15.69:0/857115728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29900b8ca0 0x7f2990163fa0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2990075f80 tx=0x7f298002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ae79ae84b73192 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.804+0000 7f298effd640 1 -- 172.21.15.69:0/857115728 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2980037070 con 0x7f29900b8ca0 2024-04-25T10:59:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.804+0000 7f298effd640 1 -- 172.21.15.69:0/857115728 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2980032440 con 0x7f29900b8ca0 2024-04-25T10:59:23.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.804+0000 7f298effd640 1 -- 172.21.15.69:0/857115728 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2980032810 con 0x7f29900b8ca0 2024-04-25T10:59:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.804+0000 7f29962f2640 1 -- 172.21.15.69:0/857115728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29900b8ca0 msgr2=0x7f2990163fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.804+0000 7f29962f2640 1 --2- 172.21.15.69:0/857115728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29900b8ca0 0x7f2990163fa0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2990075f80 tx=0x7f298002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.804+0000 7f29962f2640 1 -- 172.21.15.69:0/857115728 shutdown_connections 2024-04-25T10:59:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.804+0000 7f29962f2640 1 --2- 172.21.15.69:0/857115728 >> v2:172.21.15.146:3300/0 conn(0x7f29900d1f30 0x7f29900d2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.804+0000 7f29962f2640 1 --2- 172.21.15.69:0/857115728 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f29901644e0 0x7f29901648f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.804+0000 7f29962f2640 1 --2- 172.21.15.69:0/857115728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29900b8ca0 0x7f2990163fa0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.804+0000 7f29962f2640 1 -- 172.21.15.69:0/857115728 >> 172.21.15.69:0/857115728 conn(0x7f29900d3aa0 msgr2=0x7f29900d3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.804+0000 7f29962f2640 1 -- 172.21.15.69:0/857115728 shutdown_connections 2024-04-25T10:59:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.804+0000 7f29962f2640 1 -- 172.21.15.69:0/857115728 wait complete. 2024-04-25T10:59:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.806+0000 7f29962f2640 1 Processor -- start 2024-04-25T10:59:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.806+0000 7f29962f2640 1 -- start start 2024-04-25T10:59:23.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.808+0000 7f29962f2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29900b8ca0 0x7f29900c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.808+0000 7f29962f2640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f29900d1f30 0x7f299014fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.808+0000 7f29962f2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29901644e0 0x7f29900c7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.808+0000 7f29962f2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f299006e280 con 0x7f29901644e0 2024-04-25T10:59:23.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.808+0000 7f29962f2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f29900d32c0 con 0x7f29900b8ca0 2024-04-25T10:59:23.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.808+0000 7f29962f2640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f299006e560 con 0x7f29900d1f30 2024-04-25T10:59:23.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.808+0000 7f299488f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29901644e0 0x7f29900c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.808+0000 7f299488f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29901644e0 0x7f29900c7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41246/0 (socket says 172.21.15.69:41246) 2024-04-25T10:59:23.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.808+0000 7f299488f640 1 -- 172.21.15.69:0/2176308391 learned_addr learned my addr 172.21.15.69:0/2176308391 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:23.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.808+0000 7f298ffff640 1 --2- 172.21.15.69:0/2176308391 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29900b8ca0 0x7f29900c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.808+0000 7f298f7fe640 1 --2- 172.21.15.69:0/2176308391 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f29900d1f30 0x7f299014fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.808+0000 7f299488f640 1 -- 172.21.15.69:0/2176308391 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f29900d1f30 msgr2=0x7f299014fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.808+0000 7f299488f640 1 --2- 172.21.15.69:0/2176308391 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f29900d1f30 0x7f299014fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.808+0000 7f299488f640 1 -- 172.21.15.69:0/2176308391 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29900b8ca0 msgr2=0x7f29900c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.808+0000 7f299488f640 1 --2- 172.21.15.69:0/2176308391 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29900b8ca0 0x7f29900c8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.808+0000 7f299488f640 1 -- 172.21.15.69:0/2176308391 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29900cc700 con 0x7f29901644e0 2024-04-25T10:59:23.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.808+0000 7f299488f640 1 --2- 172.21.15.69:0/2176308391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29901644e0 0x7f29900c7540 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f298800cd10 tx=0x7f298800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:23.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.808+0000 7f298cff9640 1 -- 172.21.15.69:0/2176308391 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298800f680 con 0x7f29901644e0 2024-04-25T10:59:23.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.808+0000 7f298cff9640 1 -- 172.21.15.69:0/2176308391 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f298800f840 con 0x7f29901644e0 2024-04-25T10:59:23.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.808+0000 7f29962f2640 1 -- 172.21.15.69:0/2176308391 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29900cc9e0 con 0x7f29901644e0 2024-04-25T10:59:23.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.808+0000 7f298cff9640 1 -- 172.21.15.69:0/2176308391 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298800fc30 con 0x7f29901644e0 2024-04-25T10:59:23.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.808+0000 7f29962f2640 1 -- 172.21.15.69:0/2176308391 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29900ccfa0 con 0x7f29901644e0 2024-04-25T10:59:23.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.810+0000 7f29962f2640 1 -- 172.21.15.69:0/2176308391 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29900cc470 con 0x7f29901644e0 2024-04-25T10:59:23.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.810+0000 7f298cff9640 1 -- 172.21.15.69:0/2176308391 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f29880213f0 con 0x7f29901644e0 2024-04-25T10:59:23.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.810+0000 7f298cff9640 1 --2- 172.21.15.69:0/2176308391 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f297c0744b0 0x7f297c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:23.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.811+0000 7f298cff9640 1 -- 172.21.15.69:0/2176308391 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2988095d60 con 0x7f29901644e0 2024-04-25T10:59:23.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.811+0000 7f298ffff640 1 --2- 172.21.15.69:0/2176308391 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f297c0744b0 0x7f297c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:23.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.811+0000 7f298ffff640 1 --2- 172.21.15.69:0/2176308391 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f297c0744b0 0x7f297c0769c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2980034000 tx=0x7f298002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:23.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.815+0000 7f298cff9640 1 -- 172.21.15.69:0/2176308391 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2988014030 con 0x7f29901644e0 2024-04-25T10:59:23.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.992+0000 7f29962f2640 1 -- 172.21.15.69:0/2176308391 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2990050740 con 0x7f29901644e0 2024-04-25T10:59:23.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.993+0000 7f298cff9640 1 -- 172.21.15.69:0/2176308391 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f298805ffc0 con 0x7f29901644e0 2024-04-25T10:59:23.994 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:23.994 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":20,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T10:59:20.630151+0000","last_up_change":"2024-04-25T10:59:05.546839+0000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:59:06.422674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:59:16.727980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:59:17.938486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6825","nonce":1876101065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6827","nonce":1876101065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6831","nonce":1876101065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6829","nonce":1876101065}]},"public_addr":"172.21.15.69:6825/1876101065","cluster_addr":"172.21.15.69:6827/1876101065","heartbeat_back_addr":"172.21.15.69:6831/1876101065","heartbeat_front_addr":"172.21.15.69:6829/1876101065","state":["exists","up"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6801","nonce":1806438563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6803","nonce":1806438563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6807","nonce":1806438563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6805","nonce":1806438563}]},"public_addr":"172.21.15.69:6801/1806438563","cluster_addr":"172.21.15.69:6803/1806438563","heartbeat_back_addr":"172.21.15.69:6807/1806438563","heartbeat_front_addr":"172.21.15.69:6805/1806438563","state":["exists","up"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6817","nonce":1593617983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6819","nonce":1593617983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6823","nonce":1593617983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6821","nonce":1593617983}]},"public_addr":"172.21.15.69:6817/1593617983","cluster_addr":"172.21.15.69:6819/1593617983","heartbeat_back_addr":"172.21.15.69:6823/1593617983","heartbeat_front_addr":"172.21.15.69:6821/1593617983","state":["exists","up"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6809","nonce":3950661294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6811","nonce":3950661294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6815","nonce":3950661294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6813","nonce":3950661294}]},"public_addr":"172.21.15.69:6809/3950661294","cluster_addr":"172.21.15.69:6811/3950661294","heartbeat_back_addr":"172.21.15.69:6815/3950661294","heartbeat_front_addr":"172.21.15.69:6813/3950661294","state":["exists","up"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6801","nonce":2508929872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6803","nonce":2508929872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6807","nonce":2508929872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6805","nonce":2508929872}]},"public_addr":"172.21.15.146:6801/2508929872","cluster_addr":"172.21.15.146:6803/2508929872","heartbeat_back_addr":"172.21.15.146:6807/2508929872","heartbeat_front_addr":"172.21.15.146:6805/2508929872","state":["exists","up"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6811","nonce":1310911138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6816","nonce":1310911138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6823","nonce":1310911138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6821","nonce":1310911138}]},"public_addr":"172.21.15.146:6811/1310911138","cluster_addr":"172.21.15.146:6816/1310911138","heartbeat_back_addr":"172.21.15.146:6823/1310911138","heartbeat_front_addr":"172.21.15.146:6821/1310911138","state":["exists","up"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6810","nonce":634688021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6813","nonce":634688021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6820","nonce":634688021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6817","nonce":634688021}]},"public_addr":"172.21.15.146:6810/634688021","cluster_addr":"172.21.15.146:6813/634688021","heartbeat_back_addr":"172.21.15.146:6820/634688021","heartbeat_front_addr":"172.21.15.146:6817/634688021","state":["exists","up"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6825","nonce":3814215935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6827","nonce":3814215935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6831","nonce":3814215935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6829","nonce":3814215935}]},"public_addr":"172.21.15.146:6825/3814215935","cluster_addr":"172.21.15.146:6827/3814215935","heartbeat_back_addr":"172.21.15.146:6831/3814215935","heartbeat_front_addr":"172.21.15.146:6829/3814215935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.820200+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.850189+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.764560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.866248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.807897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.869881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.782310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.827131+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:59:23.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.995+0000 7f29962f2640 1 -- 172.21.15.69:0/2176308391 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f297c0744b0 msgr2=0x7f297c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.995+0000 7f29962f2640 1 --2- 172.21.15.69:0/2176308391 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f297c0744b0 0x7f297c0769c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2980034000 tx=0x7f298002e000 comp rx=0 tx=0).stop 2024-04-25T10:59:23.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.995+0000 7f29962f2640 1 -- 172.21.15.69:0/2176308391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29901644e0 msgr2=0x7f29900c7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:23.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.995+0000 7f29962f2640 1 --2- 172.21.15.69:0/2176308391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29901644e0 0x7f29900c7540 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f298800cd10 tx=0x7f298800a650 comp rx=0 tx=0).stop 2024-04-25T10:59:23.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.995+0000 7f29962f2640 1 -- 172.21.15.69:0/2176308391 shutdown_connections 2024-04-25T10:59:23.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.995+0000 7f29962f2640 1 --2- 172.21.15.69:0/2176308391 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f297c0744b0 0x7f297c0769c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.995+0000 7f29962f2640 1 --2- 172.21.15.69:0/2176308391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29901644e0 0x7f29900c7540 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.995+0000 7f29962f2640 1 --2- 172.21.15.69:0/2176308391 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f29900d1f30 0x7f299014fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.995+0000 7f29962f2640 1 --2- 172.21.15.69:0/2176308391 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29900b8ca0 0x7f29900c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:23.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.995+0000 7f29962f2640 1 -- 172.21.15.69:0/2176308391 >> 172.21.15.69:0/2176308391 conn(0x7f29900d3aa0 msgr2=0x7f299015ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:23.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.995+0000 7f29962f2640 1 -- 172.21.15.69:0/2176308391 shutdown_connections 2024-04-25T10:59:23.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:23.995+0000 7f29962f2640 1 -- 172.21.15.69:0/2176308391 wait complete. 2024-04-25T10:59:24.008 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T10:59:24.008 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T10:59:24.009 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T10:59:24.009 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T10:59:24.010 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T10:59:24.010 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T10:59:24.010 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T10:59:24.011 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T10:59:24.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.215+0000 7f8299ef9640 1 Processor -- start 2024-04-25T10:59:24.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.216+0000 7f8299ef9640 1 -- start start 2024-04-25T10:59:24.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.218+0000 7fd5d4d14640 1 Processor -- start 2024-04-25T10:59:24.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.218+0000 7f8299ef9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f828c047db0 0x7f828c0e7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.218+0000 7f8299ef9640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f828c04f5d0 con 0x7f828c034060 2024-04-25T10:59:24.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.218+0000 7f8299ef9640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f828c04ff50 con 0x7f828c048300 2024-04-25T10:59:24.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.218+0000 7f8299ef9640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f828c0530c0 con 0x7f828c047db0 2024-04-25T10:59:24.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.218+0000 7fd5d4d14640 1 -- start start 2024-04-25T10:59:24.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.218+0000 7f8292ffd640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f828c034060 0x7f828c053550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:57160/0 (socket says 172.21.15.69:57160) 2024-04-25T10:59:24.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.218+0000 7f8293fff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f828c047db0 0x7f828c0e7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.218+0000 7f8293fff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f828c047db0 0x7f828c0e7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.69:58126/0 (socket says 172.21.15.69:58126) 2024-04-25T10:59:24.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.218+0000 7f8293fff640 1 -- 172.21.15.69:0/2319361579 learned_addr learned my addr 172.21.15.69:0/2319361579 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:24.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.219+0000 7f82927fc640 1 -- 172.21.15.69:0/2319361579 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2089859310 0 0) 0x7f8280001110 con 0x7f828c034060 2024-04-25T10:59:24.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.219+0000 7f82927fc640 1 -- 172.21.15.69:0/2319361579 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8284003520 con 0x7f828c034060 2024-04-25T10:59:24.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.219+0000 7f82927fc640 1 -- 172.21.15.69:0/2319361579 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1022096357 0 0) 0x7f828c04ff50 con 0x7f828c048300 2024-04-25T10:59:24.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.219+0000 7f8293fff640 1 -- 172.21.15.69:0/2319361579 >> v1:172.21.15.146:6789/0 conn(0x7f828c048300 legacy=0x7f828c033950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.220+0000 7f8293fff640 1 -- 172.21.15.69:0/2319361579 >> v1:172.21.15.69:6789/0 conn(0x7f828c034060 legacy=0x7f828c053550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.220+0000 7f8293fff640 1 -- 172.21.15.69:0/2319361579 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f828c050230 con 0x7f828c047db0 2024-04-25T10:59:24.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.220+0000 7fd05a6ca640 1 Processor -- start 2024-04-25T10:59:24.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.220+0000 7fd5d4d14640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5d014ad60 0x7fd5d0070010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.220+0000 7fd5d4d14640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fd5d00b5e40 0x7fd5d00b6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.220+0000 7fd5d4d14640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd5d01681d0 0x7fd5d01685e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.220+0000 7fd5d4d14640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fd5d00b6790 con 0x7fd5d00b5e40 2024-04-25T10:59:24.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.220+0000 7fd5d4d14640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fd5d00b4af0 con 0x7fd5d01681d0 2024-04-25T10:59:24.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.220+0000 7fd5d4d14640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd5d00b4dd0 con 0x7fd5d014ad60 2024-04-25T10:59:24.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.219+0000 7f4a6b37d640 1 Processor -- start 2024-04-25T10:59:24.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.219+0000 7f4a6b37d640 1 -- start start 2024-04-25T10:59:24.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.220+0000 7fd05a6ca640 1 -- start start 2024-04-25T10:59:24.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.220+0000 7fd5cdd9b640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fd5d00b5e40 0x7fd5d00b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.221+0000 7fd5cdd9b640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fd5d00b5e40 0x7fd5d00b6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41250/0 (socket says 172.21.15.69:41250) 2024-04-25T10:59:24.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.221+0000 7fd5ce59c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5d014ad60 0x7fd5d0070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.221+0000 7fd5ced9d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd5d01681d0 0x7fd5d01685e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.221+0000 7fd5ced9d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd5d01681d0 0x7fd5d01685e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.69:51746/0 (socket says 172.21.15.69:51746) 2024-04-25T10:59:24.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.221+0000 7fd5ced9d640 1 -- 172.21.15.69:0/1646811007 learned_addr learned my addr 172.21.15.69:0/1646811007 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:24.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.221+0000 7f8293fff640 1 --2- 172.21.15.69:0/2319361579 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f828c047db0 0x7f828c0e7d30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f829406b8e0 tx=0x7f82940695d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5041053478aac7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.221+0000 7f82927fc640 1 -- 172.21.15.69:0/2319361579 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82940b4070 con 0x7f828c047db0 2024-04-25T10:59:24.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.221+0000 7f82927fc640 1 -- 172.21.15.69:0/2319361579 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82940a7a30 con 0x7f828c047db0 2024-04-25T10:59:24.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.221+0000 7f82927fc640 1 -- 172.21.15.69:0/2319361579 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82800017e0 con 0x7f828c047db0 2024-04-25T10:59:24.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.221+0000 7fd5ced9d640 1 -- 172.21.15.69:0/1646811007 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5d014ad60 msgr2=0x7fd5d0070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:24.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.221+0000 7fd5ced9d640 1 --2- 172.21.15.69:0/1646811007 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5d014ad60 0x7fd5d0070010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.221+0000 7fd5ced9d640 1 -- 172.21.15.69:0/1646811007 >> v2:172.21.15.69:3300/0 conn(0x7fd5d00b5e40 msgr2=0x7fd5d00b6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:24.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.222+0000 7eff74138640 1 Processor -- start 2024-04-25T10:59:24.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.221+0000 7fd5ced9d640 1 --2- 172.21.15.69:0/1646811007 >> v2:172.21.15.69:3300/0 conn(0x7fd5d00b5e40 0x7fd5d00b6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.221+0000 7fd5ced9d640 1 -- 172.21.15.69:0/1646811007 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d00b50b0 con 0x7fd5d01681d0 2024-04-25T10:59:24.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.222+0000 7f4a6b37d640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f4a6406dce0 0x7f4a640cd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.222+0000 7f4a6b37d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4a640cde90 0x7f4a6416d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.222+0000 7f4a6b37d640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a64164300 con 0x7f4a64163ef0 2024-04-25T10:59:24.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.222+0000 7f4a6b37d640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f4a640700a0 con 0x7f4a6406dce0 2024-04-25T10:59:24.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.222+0000 7f4a6b37d640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f4a6416e300 con 0x7f4a640cde90 2024-04-25T10:59:24.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.222+0000 7eff74138640 1 -- start start 2024-04-25T10:59:24.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.222+0000 7f4a69119640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f4a64163ef0 0x7f4a6406d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:57174/0 (socket says 172.21.15.69:57174) 2024-04-25T10:59:24.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.222+0000 7f4a69119640 1 -- 172.21.15.69:0/1358054066 learned_addr learned my addr 172.21.15.69:0/1358054066 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:24.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.222+0000 7f9e86746640 1 Processor -- start 2024-04-25T10:59:24.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.222+0000 7fd05a6ca640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0541616a0 0x7fd05415ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.222+0000 7fbfbe23d640 1 Processor -- start 2024-04-25T10:59:24.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.222+0000 7f8299ef9640 1 -- 172.21.15.69:0/2319361579 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f828c047db0 msgr2=0x7f828c0e7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.222+0000 7f8299ef9640 1 --2- 172.21.15.69:0/2319361579 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f828c047db0 0x7f828c0e7d30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f829406b8e0 tx=0x7f82940695d0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.222+0000 7f8299ef9640 1 -- 172.21.15.69:0/2319361579 shutdown_connections 2024-04-25T10:59:24.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.222+0000 7f8299ef9640 1 --2- 172.21.15.69:0/2319361579 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f828c047db0 0x7f828c0e7d30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.222+0000 7f8299ef9640 1 -- 172.21.15.69:0/2319361579 >> 172.21.15.69:0/2319361579 conn(0x7f828c054cf0 msgr2=0x7f828c055100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:24.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.222+0000 7f8299ef9640 1 -- 172.21.15.69:0/2319361579 shutdown_connections 2024-04-25T10:59:24.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.222+0000 7f8299ef9640 1 -- 172.21.15.69:0/2319361579 wait complete. 2024-04-25T10:59:24.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.222+0000 7fd05a6ca640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd05406e030 0x7fd05406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.222+0000 7fd05a6ca640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fd054163fc0 0x7fd0541643d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.223+0000 7fbfbe23d640 1 -- start start 2024-04-25T10:59:24.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.223+0000 7fd5ced9d640 1 --2- 172.21.15.69:0/1646811007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5d01681d0 0x7fd5d01685e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd5d00ce2f0 tx=0x7fd5c002a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4805afc6ba98b35a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.222+0000 7fd05a6ca640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fd054160980 con 0x7fd054163fc0 2024-04-25T10:59:24.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.222+0000 7fd05a6ca640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fd05406e980 con 0x7fd05406e030 2024-04-25T10:59:24.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.222+0000 7fd05a6ca640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd054164c80 con 0x7fd0541616a0 2024-04-25T10:59:24.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.223+0000 7fd0537fe640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd05406e030 0x7fd05406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.223+0000 7fd058c67640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fd054163fc0 0x7fd0541643d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.223+0000 7fd5cd59a640 1 -- 172.21.15.69:0/1646811007 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0036070 con 0x7fd5d01681d0 2024-04-25T10:59:24.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.223+0000 7fd5cd59a640 1 -- 172.21.15.69:0/1646811007 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5c0031600 con 0x7fd5d01681d0 2024-04-25T10:59:24.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.223+0000 7fd053fff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0541616a0 0x7fd05415ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.223+0000 7fd0537fe640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd05406e030 0x7fd05406e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.69:51758/0 (socket says 172.21.15.69:51758) 2024-04-25T10:59:24.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.223+0000 7fd0537fe640 1 -- 172.21.15.69:0/1686385757 learned_addr learned my addr 172.21.15.69:0/1686385757 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:24.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.223+0000 7fd0537fe640 1 -- 172.21.15.69:0/1686385757 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0541616a0 msgr2=0x7fd05415ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:24.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.223+0000 7fd0537fe640 1 --2- 172.21.15.69:0/1686385757 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0541616a0 0x7fd05415ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.223+0000 7fd0537fe640 1 -- 172.21.15.69:0/1686385757 >> v2:172.21.15.69:3300/0 conn(0x7fd054163fc0 msgr2=0x7fd0541643d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:24.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.223+0000 7fd0537fe640 1 --2- 172.21.15.69:0/1686385757 >> v2:172.21.15.69:3300/0 conn(0x7fd054163fc0 0x7fd0541643d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.223+0000 7fd0537fe640 1 -- 172.21.15.69:0/1686385757 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd054160bc0 con 0x7fd05406e030 2024-04-25T10:59:24.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.225+0000 7fd0537fe640 1 --2- 172.21.15.69:0/1686385757 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd05406e030 0x7fd05406e440 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd05408d370 tx=0x7fd04802a200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f677fb0ca434080 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.224+0000 7eff74138640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7eff6c06fc40 0x7eff6c15de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.225+0000 7eff74138640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7eff6c0aeb80 0x7eff6c0aef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.225+0000 7eff74138640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff6c070a40 con 0x7eff6c0af4d0 2024-04-25T10:59:24.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.225+0000 7eff74138640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7eff6c06c910 con 0x7eff6c06fc40 2024-04-25T10:59:24.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.225+0000 7eff74138640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7eff6c06c5c0 con 0x7eff6c0aeb80 2024-04-25T10:59:24.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.222+0000 7f4a6991a640 1 --2- 172.21.15.69:0/1358054066 >> v2:172.21.15.146:3300/0 conn(0x7f4a640cde90 0x7f4a6416d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.222+0000 7f4a68918640 1 --2- 172.21.15.69:0/1358054066 >> v2:172.21.15.69:3300/0 conn(0x7f4a6406dce0 0x7f4a640cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.223+0000 7f4a6991a640 1 -- 172.21.15.69:0/1358054066 >> v2:172.21.15.69:3300/0 conn(0x7f4a6406dce0 msgr2=0x7f4a640cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:24.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.223+0000 7f4a6991a640 1 --2- 172.21.15.69:0/1358054066 >> v2:172.21.15.69:3300/0 conn(0x7f4a6406dce0 0x7f4a640cd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.223+0000 7f4a6991a640 1 -- 172.21.15.69:0/1358054066 >> v1:172.21.15.69:6789/0 conn(0x7f4a64163ef0 legacy=0x7f4a6406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.224+0000 7f4a6991a640 1 -- 172.21.15.69:0/1358054066 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a640b4a10 con 0x7f4a640cde90 2024-04-25T10:59:24.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.224+0000 7f4a68918640 1 --2- 172.21.15.69:0/1358054066 >> v2:172.21.15.69:3300/0 conn(0x7f4a6406dce0 0x7f4a640cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:59:24.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.223+0000 7f9e86746640 1 -- start start 2024-04-25T10:59:24.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.225+0000 7fd5d4d14640 1 -- 172.21.15.69:0/1646811007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5d01681d0 msgr2=0x7fd5d01685e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.225+0000 7fd5d4d14640 1 --2- 172.21.15.69:0/1646811007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5d01681d0 0x7fd5d01685e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd5d00ce2f0 tx=0x7fd5c002a620 comp rx=0 tx=0).stop 2024-04-25T10:59:24.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.225+0000 7eff716d3640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7eff6c0aeb80 0x7eff6c0aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.225+0000 7fbfbe23d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbfb004fd90 0x7fbfb0038560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.225+0000 7fbfbe23d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfb00e73c0 0x7fbfb0007260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.225+0000 7fbfbe23d640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fbfb00077a0 0x7fbfb0054150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.225+0000 7fbfbe23d640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fbfb0006dd0 con 0x7fbfb00077a0 2024-04-25T10:59:24.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.225+0000 7f4a6991a640 1 --2- 172.21.15.69:0/1358054066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a640cde90 0x7f4a6416d540 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4a64073e70 tx=0x7f4a5c02a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=893d2a09f834e58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.225+0000 7fbfbe23d640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fbfb00061c0 con 0x7fbfb004fd90 2024-04-25T10:59:24.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.225+0000 7fbfbe23d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbfb00064a0 con 0x7fbfb00e73c0 2024-04-25T10:59:24.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.225+0000 7fd9650d1640 1 Processor -- start 2024-04-25T10:59:24.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.225+0000 7eff716d3640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7eff6c0aeb80 0x7eff6c0aef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.69:51774/0 (socket says 172.21.15.69:51774) 2024-04-25T10:59:24.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.225+0000 7eff716d3640 1 -- 172.21.15.69:0/187896622 learned_addr learned my addr 172.21.15.69:0/187896622 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:24.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.225+0000 7eff71ed4640 1 --2- 172.21.15.69:0/187896622 >> v2:172.21.15.69:3300/0 conn(0x7eff6c06fc40 0x7eff6c15de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.225+0000 7fbfbda3c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fbfb00077a0 0x7fbfb0054150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.225+0000 7fbfbca3a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfb00e73c0 0x7fbfb0007260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.225+0000 7fbfbca3a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfb00e73c0 0x7fbfb0007260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.69:58158/0 (socket says 172.21.15.69:58158) 2024-04-25T10:59:24.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.225+0000 7fbfbca3a640 1 -- 172.21.15.69:0/1368535307 learned_addr learned my addr 172.21.15.69:0/1368535307 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:24.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.225+0000 7f8299ef9640 1 Processor -- start 2024-04-25T10:59:24.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.226+0000 7eff716d3640 1 -- 172.21.15.69:0/187896622 >> v2:172.21.15.69:3300/0 conn(0x7eff6c06fc40 msgr2=0x7eff6c15de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:24.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.226+0000 7f9e86746640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9e800b7620 0x7f9e800b7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.226+0000 7eff716d3640 1 --2- 172.21.15.69:0/187896622 >> v2:172.21.15.69:3300/0 conn(0x7eff6c06fc40 0x7eff6c15de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.226+0000 7f9e86746640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f9e800b7f70 0x7f9e800b8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.226+0000 7fbfbd23b640 1 --2- 172.21.15.69:0/1368535307 >> v2:172.21.15.146:3300/0 conn(0x7fbfb004fd90 0x7fbfb0038560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.226+0000 7f9e86746640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e800d19c0 con 0x7f9e8008cd60 2024-04-25T10:59:24.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.226+0000 7f8299ef9640 1 -- start start 2024-04-25T10:59:24.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.226+0000 7f9e86746640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f9e800b96c0 con 0x7f9e800b7f70 2024-04-25T10:59:24.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.226+0000 7f9e86746640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f9e800b88c0 con 0x7f9e800b7620 2024-04-25T10:59:24.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.226+0000 7f9e7ffff640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f9e8008cd60 0x7f9e800d2280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:57176/0 (socket says 172.21.15.69:57176) 2024-04-25T10:59:24.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.226+0000 7f9e7ffff640 1 -- 172.21.15.69:0/3062582179 learned_addr learned my addr 172.21.15.69:0/3062582179 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:24.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.226+0000 7f9e7f7fe640 1 --2- 172.21.15.69:0/3062582179 >> v2:172.21.15.146:3300/0 conn(0x7f9e800b7620 0x7f9e800b7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.226+0000 7eff716d3640 1 -- 172.21.15.69:0/187896622 >> v1:172.21.15.146:6789/0 conn(0x7eff6c0af4d0 legacy=0x7eff6c161b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.226+0000 7eff716d3640 1 -- 172.21.15.69:0/187896622 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6c09fb50 con 0x7eff6c0aeb80 2024-04-25T10:59:24.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.226+0000 7eff71ed4640 1 --2- 172.21.15.69:0/187896622 >> v2:172.21.15.69:3300/0 conn(0x7eff6c06fc40 0x7eff6c15de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:24.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.227+0000 7eff716d3640 1 --2- 172.21.15.69:0/187896622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff6c0aeb80 0x7eff6c0aef90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7eff6c06f790 tx=0x7eff6802a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef27a052e8829871 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.226+0000 7fbfbda3c640 1 -- 172.21.15.69:0/1368535307 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfb00e73c0 msgr2=0x7fbfb0007260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:24.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.226+0000 7fbfbda3c640 1 --2- 172.21.15.69:0/1368535307 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfb00e73c0 0x7fbfb0007260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.226+0000 7fbfbda3c640 1 -- 172.21.15.69:0/1368535307 >> v2:172.21.15.146:3300/0 conn(0x7fbfb004fd90 msgr2=0x7fbfb0038560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:24.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.226+0000 7fbfbda3c640 1 --2- 172.21.15.69:0/1368535307 >> v2:172.21.15.146:3300/0 conn(0x7fbfb004fd90 0x7fbfb0038560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.226+0000 7fbfbda3c640 1 -- 172.21.15.69:0/1368535307 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb0006780 con 0x7fbfb00077a0 2024-04-25T10:59:24.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.227+0000 7fd5d4d14640 1 -- 172.21.15.69:0/1646811007 shutdown_connections 2024-04-25T10:59:24.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.227+0000 7fd5d4d14640 1 --2- 172.21.15.69:0/1646811007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5d01681d0 0x7fd5d01685e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.227+0000 7fd5d4d14640 1 --2- 172.21.15.69:0/1646811007 >> v2:172.21.15.69:3300/0 conn(0x7fd5d00b5e40 0x7fd5d00b6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.227+0000 7fd5d4d14640 1 --2- 172.21.15.69:0/1646811007 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5d014ad60 0x7fd5d0070010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.227+0000 7fd5d4d14640 1 -- 172.21.15.69:0/1646811007 >> 172.21.15.69:0/1646811007 conn(0x7fd5d00d1ae0 msgr2=0x7fd5d00d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:24.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.227+0000 7fd5d4d14640 1 -- 172.21.15.69:0/1646811007 shutdown_connections 2024-04-25T10:59:24.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.226+0000 7f4a5bfff640 1 -- 172.21.15.69:0/1358054066 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c007740 con 0x7f4a640cde90 2024-04-25T10:59:24.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.226+0000 7f4a5bfff640 1 -- 172.21.15.69:0/1358054066 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a5c0078e0 con 0x7f4a640cde90 2024-04-25T10:59:24.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.226+0000 7f4a5bfff640 1 -- 172.21.15.69:0/1358054066 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c007cb0 con 0x7f4a640cde90 2024-04-25T10:59:24.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.226+0000 7f9e7effd640 1 -- 172.21.15.69:0/3062582179 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1736344723 0 0) 0x7f9e800d19c0 con 0x7f9e8008cd60 2024-04-25T10:59:24.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.226+0000 7f9e7f7fe640 1 -- 172.21.15.69:0/3062582179 >> v2:172.21.15.69:3300/0 conn(0x7f9e800b7f70 msgr2=0x7f9e800b8380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:59:24.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.226+0000 7f9e7f7fe640 1 --2- 172.21.15.69:0/3062582179 >> v2:172.21.15.69:3300/0 conn(0x7f9e800b7f70 0x7f9e800b8380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.226+0000 7f9e7f7fe640 1 -- 172.21.15.69:0/3062582179 >> v1:172.21.15.69:6789/0 conn(0x7f9e8008cd60 legacy=0x7f9e800d2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.226+0000 7f9e7f7fe640 1 -- 172.21.15.69:0/3062582179 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e800d1c80 con 0x7f9e800b7620 2024-04-25T10:59:24.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.225+0000 7fd9650d1640 1 -- start start 2024-04-25T10:59:24.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.227+0000 7f9e7f7fe640 1 --2- 172.21.15.69:0/3062582179 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e800b7620 0x7f9e800b7a30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9e800713b0 tx=0x7f9e7402a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97cd55d27447ed9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.227+0000 7f4a6b37d640 1 -- 172.21.15.69:0/1358054066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a640cde90 msgr2=0x7f4a6416d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.227+0000 7f4a6b37d640 1 --2- 172.21.15.69:0/1358054066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a640cde90 0x7f4a6416d540 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4a64073e70 tx=0x7f4a5c02a680 comp rx=0 tx=0).stop 2024-04-25T10:59:24.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.227+0000 7fd5d4d14640 1 -- 172.21.15.69:0/1646811007 wait complete. 2024-04-25T10:59:24.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7f9e7effd640 1 -- 172.21.15.69:0/3062582179 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e74037070 con 0x7f9e800b7620 2024-04-25T10:59:24.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7fbfbda3c640 1 --2- 172.21.15.69:0/1368535307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfb00077a0 0x7fbfb0054150 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0011dc0 tx=0x7fbfb402a010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37437433fdf6ef04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7f9e7effd640 1 -- 172.21.15.69:0/3062582179 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e74032440 con 0x7f9e800b7620 2024-04-25T10:59:24.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7eff70ed2640 1 -- 172.21.15.69:0/187896622 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff68037070 con 0x7eff6c0aeb80 2024-04-25T10:59:24.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7eff70ed2640 1 -- 172.21.15.69:0/187896622 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff6802acf0 con 0x7eff6c0aeb80 2024-04-25T10:59:24.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7eff70ed2640 1 -- 172.21.15.69:0/187896622 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff68032660 con 0x7eff6c0aeb80 2024-04-25T10:59:24.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7f9e7effd640 1 -- 172.21.15.69:0/3062582179 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e74032810 con 0x7f9e800b7620 2024-04-25T10:59:24.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.227+0000 7fd052ffd640 1 -- 172.21.15.69:0/1686385757 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd048037070 con 0x7fd05406e030 2024-04-25T10:59:24.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.227+0000 7fd052ffd640 1 -- 172.21.15.69:0/1686385757 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd048032440 con 0x7fd05406e030 2024-04-25T10:59:24.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.227+0000 7fd052ffd640 1 -- 172.21.15.69:0/1686385757 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0480327f0 con 0x7fd05406e030 2024-04-25T10:59:24.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7f8299ef9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f828c047db0 0x7f828c0ea3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7fd9650d1640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fd9600b5e40 0x7fd9600b6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7fd9650d1640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd96006c5c0 0x7fd96006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7f8299ef9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f828c048300 0x7f828c039350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7fd9650d1640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9600b6790 con 0x7fd96014ad60 2024-04-25T10:59:24.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7fd9650d1640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fd9600b4af0 con 0x7fd9600b5e40 2024-04-25T10:59:24.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7fd9650d1640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd9600b4dd0 con 0x7fd96006c5c0 2024-04-25T10:59:24.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7f4a6b37d640 1 -- 172.21.15.69:0/1358054066 shutdown_connections 2024-04-25T10:59:24.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7f4a6b37d640 1 --2- 172.21.15.69:0/1358054066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a640cde90 0x7f4a6416d540 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7fbfaffff640 1 -- 172.21.15.69:0/1368535307 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4037070 con 0x7fbfb00077a0 2024-04-25T10:59:24.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7f4a6b37d640 1 --2- 172.21.15.69:0/1358054066 >> v2:172.21.15.69:3300/0 conn(0x7f4a6406dce0 0x7f4a640cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7f8299ef9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f828c039890 0x7f828c04cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7f4a6b37d640 1 -- 172.21.15.69:0/1358054066 >> 172.21.15.69:0/1358054066 conn(0x7f4a64073050 msgr2=0x7f4a64073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:24.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7fbfaffff640 1 -- 172.21.15.69:0/1368535307 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfb402ad60 con 0x7fbfb00077a0 2024-04-25T10:59:24.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7f82937fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f828c047db0 0x7f828c0ea3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7f82937fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f828c047db0 0x7f828c0ea3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.69:51788/0 (socket says 172.21.15.69:51788) 2024-04-25T10:59:24.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7f8292ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f828c048300 0x7f828c039350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7f82937fe640 1 -- 172.21.15.69:0/1355111125 learned_addr learned my addr 172.21.15.69:0/1355111125 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:24.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7f8292ffd640 1 --2- 172.21.15.69:0/1355111125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f828c048300 0x7f828c039350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41302/0 (socket says 172.21.15.69:41302) 2024-04-25T10:59:24.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7fd05a6ca640 1 -- 172.21.15.69:0/1686385757 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd05406e030 msgr2=0x7fd05406e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7fd05a6ca640 1 --2- 172.21.15.69:0/1686385757 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd05406e030 0x7fd05406e440 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd05408d370 tx=0x7fd04802a200 comp rx=0 tx=0).stop 2024-04-25T10:59:24.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7f8293fff640 1 --2- 172.21.15.69:0/1355111125 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f828c039890 0x7f828c04cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7f8299ef9640 1 -- 172.21.15.69:0/1355111125 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f828c050970 con 0x7f828c048300 2024-04-25T10:59:24.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7f8299ef9640 1 -- 172.21.15.69:0/1355111125 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f828c0507f0 con 0x7f828c047db0 2024-04-25T10:59:24.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7fd95ed9d640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fd96014ad60 0x7fd960070010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.69:50386/0 (socket says 172.21.15.69:50386) 2024-04-25T10:59:24.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7fd95ed9d640 1 -- 172.21.15.69:0/3641607917 learned_addr learned my addr 172.21.15.69:0/3641607917 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:24.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7f8299ef9640 1 -- 172.21.15.69:0/1355111125 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f828c053c60 con 0x7f828c039890 2024-04-25T10:59:24.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7fd95e59c640 1 --2- 172.21.15.69:0/3641607917 >> v2:172.21.15.69:3300/0 conn(0x7fd9600b5e40 0x7fd9600b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.228+0000 7fd95f59e640 1 --2- 172.21.15.69:0/3641607917 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd96006c5c0 0x7fd96006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7fd05a6ca640 1 -- 172.21.15.69:0/1686385757 shutdown_connections 2024-04-25T10:59:24.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7fd05a6ca640 1 --2- 172.21.15.69:0/1686385757 >> v2:172.21.15.69:3300/0 conn(0x7fd054163fc0 0x7fd0541643d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7fd05a6ca640 1 --2- 172.21.15.69:0/1686385757 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd05406e030 0x7fd05406e440 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7fd05a6ca640 1 --2- 172.21.15.69:0/1686385757 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0541616a0 0x7fd05415ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7fd05a6ca640 1 -- 172.21.15.69:0/1686385757 >> 172.21.15.69:0/1686385757 conn(0x7fd0540d27c0 msgr2=0x7fd0540d2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:24.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7fd05a6ca640 1 -- 172.21.15.69:0/1686385757 shutdown_connections 2024-04-25T10:59:24.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7fd05a6ca640 1 -- 172.21.15.69:0/1686385757 wait complete. 2024-04-25T10:59:24.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7fd95f59e640 1 -- 172.21.15.69:0/3641607917 >> v2:172.21.15.69:3300/0 conn(0x7fd9600b5e40 msgr2=0x7fd9600b6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:24.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7fd95f59e640 1 --2- 172.21.15.69:0/3641607917 >> v2:172.21.15.69:3300/0 conn(0x7fd9600b5e40 0x7fd9600b6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7fd95f59e640 1 -- 172.21.15.69:0/3641607917 >> v1:172.21.15.146:6789/0 conn(0x7fd96014ad60 legacy=0x7fd960070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7f9e86746640 1 -- 172.21.15.69:0/3062582179 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e800b7620 msgr2=0x7f9e800b7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7f9e86746640 1 --2- 172.21.15.69:0/3062582179 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e800b7620 0x7f9e800b7a30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9e800713b0 tx=0x7f9e7402a130 comp rx=0 tx=0).stop 2024-04-25T10:59:24.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7eff74138640 1 -- 172.21.15.69:0/187896622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff6c0aeb80 msgr2=0x7eff6c0aef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7eff74138640 1 --2- 172.21.15.69:0/187896622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff6c0aeb80 0x7eff6c0aef90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7eff6c06f790 tx=0x7eff6802a060 comp rx=0 tx=0).stop 2024-04-25T10:59:24.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7fd95f59e640 1 -- 172.21.15.69:0/3641607917 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9600b50b0 con 0x7fd96006c5c0 2024-04-25T10:59:24.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7eff74138640 1 -- 172.21.15.69:0/187896622 shutdown_connections 2024-04-25T10:59:24.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7eff74138640 1 --2- 172.21.15.69:0/187896622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff6c0aeb80 0x7eff6c0aef90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7eff74138640 1 --2- 172.21.15.69:0/187896622 >> v2:172.21.15.69:3300/0 conn(0x7eff6c06fc40 0x7eff6c15de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7eff74138640 1 -- 172.21.15.69:0/187896622 >> 172.21.15.69:0/187896622 conn(0x7eff6c0d1ed0 msgr2=0x7eff6c0d22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:24.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7fbfaffff640 1 -- 172.21.15.69:0/1368535307 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4032640 con 0x7fbfb00077a0 2024-04-25T10:59:24.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7f82937fe640 1 -- 172.21.15.69:0/1355111125 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f828c039890 msgr2=0x7f828c04cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7f82937fe640 1 --2- 172.21.15.69:0/1355111125 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f828c039890 0x7f828c04cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7f82937fe640 1 -- 172.21.15.69:0/1355111125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f828c048300 msgr2=0x7f828c039350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7f82937fe640 1 --2- 172.21.15.69:0/1355111125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f828c048300 0x7f828c039350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7f82937fe640 1 -- 172.21.15.69:0/1355111125 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f828c039ca0 con 0x7f828c047db0 2024-04-25T10:59:24.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7f9e86746640 1 -- 172.21.15.69:0/3062582179 shutdown_connections 2024-04-25T10:59:24.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7f9e86746640 1 --2- 172.21.15.69:0/3062582179 >> v2:172.21.15.69:3300/0 conn(0x7f9e800b7f70 0x7f9e800b8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7f9e86746640 1 --2- 172.21.15.69:0/3062582179 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e800b7620 0x7f9e800b7a30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7f9e86746640 1 -- 172.21.15.69:0/3062582179 >> 172.21.15.69:0/3062582179 conn(0x7f9e800b5640 msgr2=0x7f9e800b5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:24.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7fd95e59c640 1 --2- 172.21.15.69:0/3641607917 >> v2:172.21.15.69:3300/0 conn(0x7fd9600b5e40 0x7fd9600b6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:24.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7eff74138640 1 -- 172.21.15.69:0/187896622 shutdown_connections 2024-04-25T10:59:24.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7f8292ffd640 1 --2- 172.21.15.69:0/1355111125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f828c048300 0x7f828c039350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:24.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7f82937fe640 1 --2- 172.21.15.69:0/1355111125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f828c047db0 0x7f828c0ea3f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f828800c480 tx=0x7f828800c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7f9e86746640 1 -- 172.21.15.69:0/3062582179 shutdown_connections 2024-04-25T10:59:24.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7eff74138640 1 -- 172.21.15.69:0/187896622 wait complete. 2024-04-25T10:59:24.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.230+0000 7f827ffff640 1 -- 172.21.15.69:0/1355111125 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8288005480 con 0x7f828c047db0 2024-04-25T10:59:24.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.230+0000 7f827ffff640 1 -- 172.21.15.69:0/1355111125 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82880035f0 con 0x7f828c047db0 2024-04-25T10:59:24.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.230+0000 7fbfbe23d640 1 -- 172.21.15.69:0/1368535307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfb00077a0 msgr2=0x7fbfb0054150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.230+0000 7fbfbe23d640 1 --2- 172.21.15.69:0/1368535307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfb00077a0 0x7fbfb0054150 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0011dc0 tx=0x7fbfb402a010 comp rx=0 tx=0).stop 2024-04-25T10:59:24.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.229+0000 7f9e86746640 1 -- 172.21.15.69:0/3062582179 wait complete. 2024-04-25T10:59:24.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.230+0000 7fbfbe23d640 1 -- 172.21.15.69:0/1368535307 shutdown_connections 2024-04-25T10:59:24.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.230+0000 7fbfbe23d640 1 --2- 172.21.15.69:0/1368535307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfb00077a0 0x7fbfb0054150 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.230+0000 7fbfbe23d640 1 --2- 172.21.15.69:0/1368535307 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfb00e73c0 0x7fbfb0007260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.230+0000 7fbfbe23d640 1 --2- 172.21.15.69:0/1368535307 >> v2:172.21.15.146:3300/0 conn(0x7fbfb004fd90 0x7fbfb0038560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.230+0000 7fbfbe23d640 1 -- 172.21.15.69:0/1368535307 >> 172.21.15.69:0/1368535307 conn(0x7fbfb003ac70 msgr2=0x7fbfb003b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:24.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.230+0000 7fbfbe23d640 1 -- 172.21.15.69:0/1368535307 shutdown_connections 2024-04-25T10:59:24.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.230+0000 7f4a6b37d640 1 -- 172.21.15.69:0/1358054066 shutdown_connections 2024-04-25T10:59:24.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.230+0000 7fd95f59e640 1 --2- 172.21.15.69:0/3641607917 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd96006c5c0 0x7fd96006c9d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd9540097a0 tx=0x7fd95402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7747c58834ff40a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.230+0000 7fd5d4d14640 1 Processor -- start 2024-04-25T10:59:24.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.230+0000 7f4a6b37d640 1 -- 172.21.15.69:0/1358054066 wait complete. 2024-04-25T10:59:24.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.230+0000 7fbfbe23d640 1 -- 172.21.15.69:0/1368535307 wait complete. 2024-04-25T10:59:24.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.230+0000 7fd5d4d14640 1 -- start start 2024-04-25T10:59:24.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.231+0000 7f827ffff640 1 -- 172.21.15.69:0/1355111125 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82880039e0 con 0x7f828c047db0 2024-04-25T10:59:24.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.231+0000 7f8299ef9640 1 -- 172.21.15.69:0/1355111125 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f828c0ea930 con 0x7f828c047db0 2024-04-25T10:59:24.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.231+0000 7f8299ef9640 1 -- 172.21.15.69:0/1355111125 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f828c04c1b0 con 0x7f828c047db0 2024-04-25T10:59:24.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.232+0000 7fd05a6ca640 1 Processor -- start 2024-04-25T10:59:24.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.232+0000 7eff74138640 1 Processor -- start 2024-04-25T10:59:24.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.232+0000 7fd05a6ca640 1 -- start start 2024-04-25T10:59:24.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.232+0000 7eff74138640 1 -- start start 2024-04-25T10:59:24.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.232+0000 7f9e86746640 1 Processor -- start 2024-04-25T10:59:24.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.233+0000 7f9e86746640 1 -- start start 2024-04-25T10:59:24.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.233+0000 7fd5d4d14640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5d00b5e40 0x7fd5d0152b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.233+0000 7fbfbe23d640 1 Processor -- start 2024-04-25T10:59:24.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.233+0000 7fbfbe23d640 1 -- start start 2024-04-25T10:59:24.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.233+0000 7fd5d4d14640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5d014ad60 0x7fd5d015e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.233+0000 7fd5d4d14640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5d01681d0 0x7fd5d015e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.233+0000 7fd5d4d14640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d0070550 con 0x7fd5d00b5e40 2024-04-25T10:59:24.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.233+0000 7fd5d4d14640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d00b5670 con 0x7fd5d01681d0 2024-04-25T10:59:24.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.233+0000 7fd5ce59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5d00b5e40 0x7fd5d0152b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.233+0000 7fd5ce59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5d00b5e40 0x7fd5d0152b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41308/0 (socket says 172.21.15.69:41308) 2024-04-25T10:59:24.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.233+0000 7fd5ce59c640 1 -- 172.21.15.69:0/535421077 learned_addr learned my addr 172.21.15.69:0/535421077 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:24.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.233+0000 7fd5ced9d640 1 --2- 172.21.15.69:0/535421077 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5d01681d0 0x7fd5d015e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.233+0000 7fd5d4d14640 1 -- 172.21.15.69:0/535421077 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd5d00706d0 con 0x7fd5d014ad60 2024-04-25T10:59:24.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.233+0000 7fd5cdd9b640 1 --2- 172.21.15.69:0/535421077 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5d014ad60 0x7fd5d015e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.234+0000 7fd5ced9d640 1 -- 172.21.15.69:0/535421077 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5d014ad60 msgr2=0x7fd5d015e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.234+0000 7fd5ced9d640 1 --2- 172.21.15.69:0/535421077 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5d014ad60 0x7fd5d015e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.234+0000 7fd5ced9d640 1 -- 172.21.15.69:0/535421077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5d00b5e40 msgr2=0x7fd5d0152b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.234+0000 7fd5ced9d640 1 --2- 172.21.15.69:0/535421077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd5d00b5e40 0x7fd5d0152b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.234+0000 7fd5ced9d640 1 -- 172.21.15.69:0/535421077 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d0163560 con 0x7fd5d01681d0 2024-04-25T10:59:24.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.233+0000 7f4a6b37d640 1 Processor -- start 2024-04-25T10:59:24.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.234+0000 7f8299ef9640 1 -- 172.21.15.69:0/1355111125 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f828c0eae40 con 0x7f828c047db0 2024-04-25T10:59:24.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.234+0000 7f827ffff640 1 -- 172.21.15.69:0/1355111125 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8288003b80 con 0x7f828c047db0 2024-04-25T10:59:24.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.234+0000 7fd5ced9d640 1 --2- 172.21.15.69:0/535421077 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5d01681d0 0x7fd5d015e900 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0030000 tx=0x7fd5c0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.233+0000 7f4a6b37d640 1 -- start start 2024-04-25T10:59:24.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.235+0000 7fd05a6ca640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd05406e030 0x7fd054073100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.235+0000 7fd05a6ca640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0541616a0 0x7fd0540badf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.235+0000 7fd05a6ca640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd054163fc0 0x7fd0540b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.235+0000 7fd053fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd05406e030 0x7fd054073100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.234+0000 7eff74138640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff6c06fc40 0x7eff6c1561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.234+0000 7eff74138640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff6c0aeb80 0x7eff6c15fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.234+0000 7eff74138640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff6c0af4d0 0x7eff6c157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.234+0000 7eff74138640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c0ce9e0 con 0x7eff6c0aeb80 2024-04-25T10:59:24.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.234+0000 7eff74138640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c15e3c0 con 0x7eff6c0af4d0 2024-04-25T10:59:24.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.234+0000 7eff74138640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7eff6c0af8e0 con 0x7eff6c06fc40 2024-04-25T10:59:24.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.234+0000 7fd5beffd640 1 -- 172.21.15.69:0/535421077 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0041020 con 0x7fd5d01681d0 2024-04-25T10:59:24.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.235+0000 7fd5beffd640 1 -- 172.21.15.69:0/535421077 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5c003c470 con 0x7fd5d01681d0 2024-04-25T10:59:24.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.235+0000 7f827ffff640 1 --2- 172.21.15.69:0/1355111125 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f8278074720 0x7f8278076c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.235+0000 7fd053fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd05406e030 0x7fd054073100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.69:51798/0 (socket says 172.21.15.69:51798) 2024-04-25T10:59:24.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.235+0000 7eff716d3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff6c0aeb80 0x7eff6c15fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.235+0000 7eff716d3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff6c0aeb80 0x7eff6c15fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41318/0 (socket says 172.21.15.69:41318) 2024-04-25T10:59:24.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.231+0000 7fd95dd9b640 1 -- 172.21.15.69:0/3641607917 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd954037070 con 0x7fd96006c5c0 2024-04-25T10:59:24.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.231+0000 7fd95dd9b640 1 -- 172.21.15.69:0/3641607917 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd954032440 con 0x7fd96006c5c0 2024-04-25T10:59:24.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.232+0000 7fd95dd9b640 1 -- 172.21.15.69:0/3641607917 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd954032810 con 0x7fd96006c5c0 2024-04-25T10:59:24.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.233+0000 7fd9650d1640 1 -- 172.21.15.69:0/3641607917 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd96006c5c0 msgr2=0x7fd96006c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.233+0000 7fd9650d1640 1 --2- 172.21.15.69:0/3641607917 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd96006c5c0 0x7fd96006c9d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd9540097a0 tx=0x7fd95402a180 comp rx=0 tx=0).stop 2024-04-25T10:59:24.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.233+0000 7fd9650d1640 1 -- 172.21.15.69:0/3641607917 shutdown_connections 2024-04-25T10:59:24.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.233+0000 7fd9650d1640 1 --2- 172.21.15.69:0/3641607917 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd96006c5c0 0x7fd96006c9d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.233+0000 7fd9650d1640 1 --2- 172.21.15.69:0/3641607917 >> v2:172.21.15.69:3300/0 conn(0x7fd9600b5e40 0x7fd9600b6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.233+0000 7fd9650d1640 1 -- 172.21.15.69:0/3641607917 >> 172.21.15.69:0/3641607917 conn(0x7fd9600d1ae0 msgr2=0x7fd9600d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:24.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.233+0000 7fd9650d1640 1 -- 172.21.15.69:0/3641607917 shutdown_connections 2024-04-25T10:59:24.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.233+0000 7fd9650d1640 1 -- 172.21.15.69:0/3641607917 wait complete. 2024-04-25T10:59:24.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.235+0000 7eff716d3640 1 -- 172.21.15.69:0/2205882014 learned_addr learned my addr 172.21.15.69:0/2205882014 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:24.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.235+0000 7f4a6b37d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a640cde90 0x7f4a640c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.235+0000 7f4a6b37d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a64163ef0 0x7f4a6416afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.235+0000 7f4a6b37d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a640cd500 0x7f4a640ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.235+0000 7f4a6b37d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4a640b52b0 con 0x7f4a640cde90 2024-04-25T10:59:24.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.235+0000 7f4a6b37d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4a640b4fd0 con 0x7f4a64163ef0 2024-04-25T10:59:24.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.235+0000 7f4a6b37d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4a640ce2a0 con 0x7f4a640cd500 2024-04-25T10:59:24.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.235+0000 7eff726d5640 1 --2- 172.21.15.69:0/2205882014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff6c0af4d0 0x7eff6c157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.235+0000 7f4a69119640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a640cde90 0x7f4a640c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.235+0000 7f4a69119640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a640cde90 0x7f4a640c7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41340/0 (socket says 172.21.15.69:41340) 2024-04-25T10:59:24.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.235+0000 7f4a69119640 1 -- 172.21.15.69:0/3932877831 learned_addr learned my addr 172.21.15.69:0/3932877831 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:24.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.235+0000 7f4a68918640 1 --2- 172.21.15.69:0/3932877831 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a64163ef0 0x7f4a6416afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.235+0000 7fd5beffd640 1 -- 172.21.15.69:0/535421077 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0041020 con 0x7fd5d01681d0 2024-04-25T10:59:24.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.235+0000 7fd5d4d14640 1 -- 172.21.15.69:0/535421077 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5d0163840 con 0x7fd5d01681d0 2024-04-25T10:59:24.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.235+0000 7fd5d4d14640 1 -- 172.21.15.69:0/535421077 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5d00b2d00 con 0x7fd5d01681d0 2024-04-25T10:59:24.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.236+0000 7f4a6991a640 1 --2- 172.21.15.69:0/3932877831 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a640cd500 0x7f4a640ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.236+0000 7eff71ed4640 1 --2- 172.21.15.69:0/2205882014 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff6c06fc40 0x7eff6c1561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.236+0000 7fd5d4d14640 1 -- 172.21.15.69:0/535421077 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd5d0050740 con 0x7fd5d01681d0 2024-04-25T10:59:24.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.235+0000 7f827ffff640 1 -- 172.21.15.69:0/1355111125 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8288094c90 con 0x7f828c047db0 2024-04-25T10:59:24.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.236+0000 7f8292ffd640 1 --2- 172.21.15.69:0/1355111125 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f8278074720 0x7f8278076c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.236+0000 7f9e86746640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e8008cd60 0x7f9e800cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.236+0000 7f4a69119640 1 -- 172.21.15.69:0/3932877831 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a640cd500 msgr2=0x7f4a640ca4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.236+0000 7f4a69119640 1 --2- 172.21.15.69:0/3932877831 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a640cd500 0x7f4a640ca4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.236+0000 7f4a69119640 1 -- 172.21.15.69:0/3932877831 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a64163ef0 msgr2=0x7f4a6416afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.236+0000 7f4a69119640 1 --2- 172.21.15.69:0/3932877831 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a64163ef0 0x7f4a6416afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.236+0000 7f4a69119640 1 -- 172.21.15.69:0/3932877831 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a64169c40 con 0x7f4a640cde90 2024-04-25T10:59:24.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.236+0000 7f9e86746640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e800b7620 0x7f9e800c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.236+0000 7eff716d3640 1 -- 172.21.15.69:0/2205882014 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff6c06fc40 msgr2=0x7eff6c1561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.236+0000 7eff716d3640 1 --2- 172.21.15.69:0/2205882014 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff6c06fc40 0x7eff6c1561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.236+0000 7eff716d3640 1 -- 172.21.15.69:0/2205882014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff6c0af4d0 msgr2=0x7eff6c157e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.236+0000 7eff716d3640 1 --2- 172.21.15.69:0/2205882014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff6c0af4d0 0x7eff6c157e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.236+0000 7eff716d3640 1 -- 172.21.15.69:0/2205882014 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6c156a20 con 0x7eff6c0aeb80 2024-04-25T10:59:24.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.236+0000 7f9e86746640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e800b7f70 0x7f9e8009d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.236+0000 7f9e86746640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8009d330 con 0x7f9e800b7f70 2024-04-25T10:59:24.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.236+0000 7f9e86746640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8009d050 con 0x7f9e800b7620 2024-04-25T10:59:24.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.236+0000 7f9e86746640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9e800d2990 con 0x7f9e8008cd60 2024-04-25T10:59:24.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.235+0000 7fd058c67640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd054163fc0 0x7fd0540b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.235+0000 7fd053fff640 1 -- 172.21.15.69:0/1566520778 learned_addr learned my addr 172.21.15.69:0/1566520778 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:24.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.236+0000 7fd0537fe640 1 --2- 172.21.15.69:0/1566520778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0541616a0 0x7fd0540badf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.236+0000 7fd05a6ca640 1 -- 172.21.15.69:0/1566520778 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd054164910 con 0x7fd054163fc0 2024-04-25T10:59:24.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.237+0000 7fd05a6ca640 1 -- 172.21.15.69:0/1566520778 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd054161180 con 0x7fd05406e030 2024-04-25T10:59:24.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.237+0000 7fd05a6ca640 1 -- 172.21.15.69:0/1566520778 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd0540d49b0 con 0x7fd0541616a0 2024-04-25T10:59:24.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.236+0000 7fd9650d1640 1 Processor -- start 2024-04-25T10:59:24.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.237+0000 7fd9650d1640 1 -- start start 2024-04-25T10:59:24.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.237+0000 7eff716d3640 1 --2- 172.21.15.69:0/2205882014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff6c0aeb80 0x7eff6c15fc10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7eff68000c00 tx=0x7eff680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.237+0000 7f9e7ffff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e8008cd60 0x7f9e800cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.237+0000 7f9e7f7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e800b7620 0x7f9e800c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.237+0000 7f9e84ce3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e800b7f70 0x7f9e8009d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.237+0000 7f9e84ce3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e800b7f70 0x7f9e8009d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41352/0 (socket says 172.21.15.69:41352) 2024-04-25T10:59:24.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.237+0000 7f9e84ce3640 1 -- 172.21.15.69:0/48053747 learned_addr learned my addr 172.21.15.69:0/48053747 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:24.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.237+0000 7f827ffff640 1 --2- 172.21.15.69:0/1355111125 >> [v2:172.21.15.146:6809/1310911138,v1:172.21.15.146:6811/1310911138] conn(0x7f827807c9f0 0x7f827807ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.237+0000 7fbfbe23d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfb00077a0 0x7fbfb00d8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.237+0000 7fbfbe23d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfb004fd90 0x7fbfb00e62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.237+0000 7fbfbe23d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbfb00e73c0 0x7fbfb00dfc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.237+0000 7f827ffff640 1 -- 172.21.15.69:0/1355111125 --> [v2:172.21.15.146:6809/1310911138,v1:172.21.15.146:6811/1310911138] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8288014e50 con 0x7f827807c9f0 2024-04-25T10:59:24.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.237+0000 7fbfbe23d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb0054690 con 0x7fbfb00077a0 2024-04-25T10:59:24.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.237+0000 7fbfbe23d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb0055930 con 0x7fbfb00e73c0 2024-04-25T10:59:24.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.237+0000 7fbfbe23d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbfb0054810 con 0x7fbfb004fd90 2024-04-25T10:59:24.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.237+0000 7f827ffff640 1 -- 172.21.15.69:0/1355111125 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f82880972f0 con 0x7f828c047db0 2024-04-25T10:59:24.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.237+0000 7f8292ffd640 1 --2- 172.21.15.69:0/1355111125 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f8278074720 0x7f8278076c30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8280001950 tx=0x7f82800018a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.237+0000 7fbfbd23b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfb00077a0 0x7fbfb00d8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.237+0000 7fbfbd23b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfb00077a0 0x7fbfb00d8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41360/0 (socket says 172.21.15.69:41360) 2024-04-25T10:59:24.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.237+0000 7fbfbd23b640 1 -- 172.21.15.69:0/757339246 learned_addr learned my addr 172.21.15.69:0/757339246 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:24.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.237+0000 7fd5beffd640 1 -- 172.21.15.69:0/535421077 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd5c0044020 con 0x7fd5d01681d0 2024-04-25T10:59:24.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.237+0000 7fd053fff640 1 -- 172.21.15.69:0/1566520778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0541616a0 msgr2=0x7fd0540badf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.237+0000 7fbfbca3a640 1 --2- 172.21.15.69:0/757339246 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfb004fd90 0x7fbfb00e62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.237+0000 7fd053fff640 1 --2- 172.21.15.69:0/1566520778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0541616a0 0x7fd0540badf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.237+0000 7fd053fff640 1 -- 172.21.15.69:0/1566520778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd054163fc0 msgr2=0x7fd0540b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.237+0000 7fd053fff640 1 --2- 172.21.15.69:0/1566520778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd054163fc0 0x7fd0540b57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.237+0000 7fbfbda3c640 1 --2- 172.21.15.69:0/757339246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbfb00e73c0 0x7fbfb00dfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.237+0000 7f4a69119640 1 --2- 172.21.15.69:0/3932877831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a640cde90 0x7f4a640c7ee0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000b520 tx=0x7f4a6000b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.237+0000 7fd053fff640 1 -- 172.21.15.69:0/1566520778 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05415a9b0 con 0x7fd05406e030 2024-04-25T10:59:24.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.238+0000 7eff5e7fc640 1 -- 172.21.15.69:0/2205882014 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff68037040 con 0x7eff6c0aeb80 2024-04-25T10:59:24.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.238+0000 7fd053fff640 1 --2- 172.21.15.69:0/1566520778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd05406e030 0x7fd054073100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd04400d650 tx=0x7fd04400db10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.238+0000 7f8293fff640 1 --2- 172.21.15.69:0/1355111125 >> [v2:172.21.15.146:6809/1310911138,v1:172.21.15.146:6811/1310911138] conn(0x7f827807c9f0 0x7f827807ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.238+0000 7eff5e7fc640 1 -- 172.21.15.69:0/2205882014 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff68004230 con 0x7eff6c0aeb80 2024-04-25T10:59:24.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.238+0000 7fd5beffd640 1 --2- 172.21.15.69:0/535421077 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd5b00748e0 0x7fd5b0076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.238+0000 7fbfbda3c640 1 -- 172.21.15.69:0/757339246 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfb004fd90 msgr2=0x7fbfb00e62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.238+0000 7fbfbda3c640 1 --2- 172.21.15.69:0/757339246 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfb004fd90 0x7fbfb00e62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.238+0000 7fbfbda3c640 1 -- 172.21.15.69:0/757339246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfb00077a0 msgr2=0x7fbfb00d8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.238+0000 7fbfbda3c640 1 --2- 172.21.15.69:0/757339246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbfb00077a0 0x7fbfb00d8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.238+0000 7fbfbda3c640 1 -- 172.21.15.69:0/757339246 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb0179db0 con 0x7fbfb00e73c0 2024-04-25T10:59:24.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.238+0000 7f9e84ce3640 1 -- 172.21.15.69:0/48053747 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e8008cd60 msgr2=0x7f9e800cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.238+0000 7f9e84ce3640 1 --2- 172.21.15.69:0/48053747 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e8008cd60 0x7f9e800cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.238+0000 7f9e84ce3640 1 -- 172.21.15.69:0/48053747 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e800b7620 msgr2=0x7f9e800c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.238+0000 7f9e84ce3640 1 --2- 172.21.15.69:0/48053747 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e800b7620 0x7f9e800c9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.238+0000 7f9e84ce3640 1 -- 172.21.15.69:0/48053747 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e800a07e0 con 0x7f9e800b7f70 2024-04-25T10:59:24.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.238+0000 7fd5beffd640 1 -- 172.21.15.69:0/535421077 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd5c0031400 con 0x7fd5d01681d0 2024-04-25T10:59:24.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.238+0000 7fd050ff9640 1 -- 172.21.15.69:0/1566520778 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd044004030 con 0x7fd05406e030 2024-04-25T10:59:24.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.238+0000 7fd5beffd640 1 --2- 172.21.15.69:0/535421077 >> [v2:172.21.15.69:6808/3950661294,v1:172.21.15.69:6809/3950661294] conn(0x7fd5b007cad0 0x7fd5b007ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.238+0000 7fd5beffd640 1 -- 172.21.15.69:0/535421077 --> [v2:172.21.15.69:6808/3950661294,v1:172.21.15.69:6809/3950661294] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd5c0033ec0 con 0x7fd5b007cad0 2024-04-25T10:59:24.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.238+0000 7fbfbda3c640 1 --2- 172.21.15.69:0/757339246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbfb00e73c0 0x7fbfb00dfc60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4031000 tx=0x7fbfb40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.238+0000 7f9e84ce3640 1 --2- 172.21.15.69:0/48053747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e800b7f70 0x7f9e8009d7f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9e7800b990 tx=0x7f9e7800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.238+0000 7fd5beffd640 1 -- 172.21.15.69:0/535421077 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd5c007c290 con 0x7fd5d01681d0 2024-04-25T10:59:24.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.238+0000 7fd05a6ca640 1 -- 172.21.15.69:0/1566520778 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd054073a00 con 0x7fd05406e030 2024-04-25T10:59:24.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.238+0000 7fd05a6ca640 1 -- 172.21.15.69:0/1566520778 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0540c9ac0 con 0x7fd05406e030 2024-04-25T10:59:24.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.238+0000 7eff5e7fc640 1 -- 172.21.15.69:0/2205882014 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff68003610 con 0x7eff6c0aeb80 2024-04-25T10:59:24.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.238+0000 7eff74138640 1 -- 172.21.15.69:0/2205882014 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff6c156d00 con 0x7eff6c0aeb80 2024-04-25T10:59:24.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.238+0000 7fbfadffb640 1 -- 172.21.15.69:0/757339246 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4040020 con 0x7fbfb00e73c0 2024-04-25T10:59:24.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.239+0000 7fbfadffb640 1 -- 172.21.15.69:0/757339246 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfb4038040 con 0x7fbfb00e73c0 2024-04-25T10:59:24.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.239+0000 7fbfadffb640 1 -- 172.21.15.69:0/757339246 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4037070 con 0x7fbfb00e73c0 2024-04-25T10:59:24.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.238+0000 7fd5ce59c640 1 --2- 172.21.15.69:0/535421077 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd5b00748e0 0x7fd5b0076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.239+0000 7fd5cdd9b640 1 --2- 172.21.15.69:0/535421077 >> [v2:172.21.15.69:6808/3950661294,v1:172.21.15.69:6809/3950661294] conn(0x7fd5b007cad0 0x7fd5b007ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.238+0000 7fd050ff9640 1 -- 172.21.15.69:0/1566520778 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0440041d0 con 0x7fd05406e030 2024-04-25T10:59:24.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.238+0000 7fd050ff9640 1 -- 172.21.15.69:0/1566520778 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0440043a0 con 0x7fd05406e030 2024-04-25T10:59:24.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.239+0000 7eff74138640 1 -- 172.21.15.69:0/2205882014 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff6c153e70 con 0x7eff6c0aeb80 2024-04-25T10:59:24.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.239+0000 7fbfbe23d640 1 -- 172.21.15.69:0/757339246 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfb00ddf30 con 0x7fbfb00e73c0 2024-04-25T10:59:24.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.239+0000 7f9e7cff9640 1 -- 172.21.15.69:0/48053747 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e78019070 con 0x7f9e800b7f70 2024-04-25T10:59:24.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.239+0000 7fd9650d1640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd96006c5c0 0x7fd96015ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.239+0000 7fd9650d1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9600b5e40 0x7fd960160410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.239+0000 7fd95ed9d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd96006c5c0 0x7fd96015ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.239+0000 7fd95ed9d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd96006c5c0 0x7fd96015ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.69:58238/0 (socket says 172.21.15.69:58238) 2024-04-25T10:59:24.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.239+0000 7fd95ed9d640 1 -- 172.21.15.69:0/1979031114 learned_addr learned my addr 172.21.15.69:0/1979031114 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:24.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.239+0000 7fd9650d1640 1 --2- 172.21.15.69:0/1979031114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd96014ad60 0x7fd9600b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.239+0000 7fbfbe23d640 1 -- 172.21.15.69:0/757339246 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfb017a500 con 0x7fbfb00e73c0 2024-04-25T10:59:24.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.239+0000 7fd9650d1640 1 -- 172.21.15.69:0/1979031114 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd96006fb80 con 0x7fd9600b5e40 2024-04-25T10:59:24.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.239+0000 7fd9650d1640 1 -- 172.21.15.69:0/1979031114 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd9600b5670 con 0x7fd96014ad60 2024-04-25T10:59:24.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.239+0000 7fd9650d1640 1 -- 172.21.15.69:0/1979031114 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd96006fd00 con 0x7fd96006c5c0 2024-04-25T10:59:24.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.239+0000 7fd05a6ca640 1 -- 172.21.15.69:0/1566520778 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd034001030 con 0x7fd05406e030 2024-04-25T10:59:24.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.240+0000 7f4a59ffb640 1 -- 172.21.15.69:0/3932877831 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6000f640 con 0x7f4a640cde90 2024-04-25T10:59:24.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.240+0000 7f4a59ffb640 1 -- 172.21.15.69:0/3932877831 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a60015020 con 0x7f4a640cde90 2024-04-25T10:59:24.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.240+0000 7f4a59ffb640 1 -- 172.21.15.69:0/3932877831 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6000f9e0 con 0x7f4a640cde90 2024-04-25T10:59:24.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.240+0000 7f4a6b37d640 1 -- 172.21.15.69:0/3932877831 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a64169f00 con 0x7f4a640cde90 2024-04-25T10:59:24.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.240+0000 7f4a6b37d640 1 -- 172.21.15.69:0/3932877831 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a6416a4c0 con 0x7f4a640cde90 2024-04-25T10:59:24.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.240+0000 7eff74138640 1 -- 172.21.15.69:0/2205882014 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7eff6c154250 con 0x7eff6c0aeb80 2024-04-25T10:59:24.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.241+0000 7fd050ff9640 1 -- 172.21.15.69:0/1566520778 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd04401b020 con 0x7fd05406e030 2024-04-25T10:59:24.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.240+0000 7fd95f59e640 1 --2- 172.21.15.69:0/1979031114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd96014ad60 0x7fd9600b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.241+0000 7fbfbe23d640 1 -- 172.21.15.69:0/757339246 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbf98001030 con 0x7fbfb00e73c0 2024-04-25T10:59:24.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.239+0000 7f9e86746640 1 -- 172.21.15.69:0/48053747 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e800a0ac0 con 0x7f9e800b7f70 2024-04-25T10:59:24.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.239+0000 7f9e86746640 1 -- 172.21.15.69:0/48053747 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e800a1080 con 0x7f9e800b7f70 2024-04-25T10:59:24.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.240+0000 7f9e7cff9640 1 -- 172.21.15.69:0/48053747 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e7800cae0 con 0x7f9e800b7f70 2024-04-25T10:59:24.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.240+0000 7f9e7cff9640 1 -- 172.21.15.69:0/48053747 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e78014440 con 0x7f9e800b7f70 2024-04-25T10:59:24.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.241+0000 7f9e86746640 1 -- 172.21.15.69:0/48053747 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9e80050740 con 0x7f9e800b7f70 2024-04-25T10:59:24.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.240+0000 7fd5ce59c640 1 --2- 172.21.15.69:0/535421077 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd5b00748e0 0x7fd5b0076df0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8006a90 tx=0x7fd5c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.241+0000 7fd95e59c640 1 --2- 172.21.15.69:0/1979031114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9600b5e40 0x7fd960160410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.241+0000 7fd95f59e640 1 -- 172.21.15.69:0/1979031114 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd96006c5c0 msgr2=0x7fd96015ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.241+0000 7fd95f59e640 1 --2- 172.21.15.69:0/1979031114 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd96006c5c0 0x7fd96015ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.241+0000 7fd95f59e640 1 -- 172.21.15.69:0/1979031114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9600b5e40 msgr2=0x7fd960160410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.241+0000 7fd95f59e640 1 --2- 172.21.15.69:0/1979031114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9600b5e40 0x7fd960160410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.241+0000 7fd95f59e640 1 -- 172.21.15.69:0/1979031114 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd96015f790 con 0x7fd96014ad60 2024-04-25T10:59:24.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.241+0000 7fd050ff9640 1 --2- 172.21.15.69:0/1566520778 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd040074580 0x7fd040076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.241+0000 7fd050ff9640 1 -- 172.21.15.69:0/1566520778 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd044094dc0 con 0x7fd05406e030 2024-04-25T10:59:24.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.242+0000 7fd050ff9640 1 --2- 172.21.15.69:0/1566520778 >> [v2:172.21.15.69:6824/1876101065,v1:172.21.15.69:6825/1876101065] conn(0x7fd04007c850 0x7fd04007eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.242+0000 7fd95f59e640 1 --2- 172.21.15.69:0/1979031114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd96014ad60 0x7fd9600b08e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd954031000 tx=0x7fd9540046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.242+0000 7fd050ff9640 1 -- 172.21.15.69:0/1566520778 --> [v2:172.21.15.69:6824/1876101065,v1:172.21.15.69:6825/1876101065] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd04401de50 con 0x7fd04007c850 2024-04-25T10:59:24.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.242+0000 7fd050ff9640 1 -- 172.21.15.69:0/1566520778 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd04409a3f0 con 0x7fd05406e030 2024-04-25T10:59:24.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.242+0000 7f8293fff640 1 --2- 172.21.15.69:0/1355111125 >> [v2:172.21.15.146:6809/1310911138,v1:172.21.15.146:6811/1310911138] conn(0x7f827807c9f0 0x7f827807ee40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.242+0000 7fd9477fe640 1 -- 172.21.15.69:0/1979031114 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd954040020 con 0x7fd96014ad60 2024-04-25T10:59:24.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.242+0000 7fd9477fe640 1 -- 172.21.15.69:0/1979031114 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd954038040 con 0x7fd96014ad60 2024-04-25T10:59:24.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.242+0000 7fd9477fe640 1 -- 172.21.15.69:0/1979031114 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd954037070 con 0x7fd96014ad60 2024-04-25T10:59:24.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.242+0000 7fd9650d1640 1 -- 172.21.15.69:0/1979031114 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd96015fa70 con 0x7fd96014ad60 2024-04-25T10:59:24.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.242+0000 7fd9650d1640 1 -- 172.21.15.69:0/1979031114 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd960148630 con 0x7fd96014ad60 2024-04-25T10:59:24.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.242+0000 7fd058c67640 1 --2- 172.21.15.69:0/1566520778 >> [v2:172.21.15.69:6824/1876101065,v1:172.21.15.69:6825/1876101065] conn(0x7fd04007c850 0x7fd04007eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.242+0000 7f4a6b37d640 1 -- 172.21.15.69:0/3932877831 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4a6416ab80 con 0x7f4a640cde90 2024-04-25T10:59:24.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.242+0000 7fd0537fe640 1 --2- 172.21.15.69:0/1566520778 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd040074580 0x7fd040076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.242+0000 7fd058c67640 1 --2- 172.21.15.69:0/1566520778 >> [v2:172.21.15.69:6824/1876101065,v1:172.21.15.69:6825/1876101065] conn(0x7fd04007c850 0x7fd04007eca0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.243+0000 7f4a59ffb640 1 -- 172.21.15.69:0/3932877831 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4a6000fb80 con 0x7f4a640cde90 2024-04-25T10:59:24.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.242+0000 7fbfadffb640 1 -- 172.21.15.69:0/757339246 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbfb4043020 con 0x7fbfb00e73c0 2024-04-25T10:59:24.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.243+0000 7fbfadffb640 1 --2- 172.21.15.69:0/757339246 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fbf9c0745b0 0x7fbf9c076ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.243+0000 7fd5cdd9b640 1 --2- 172.21.15.69:0/535421077 >> [v2:172.21.15.69:6808/3950661294,v1:172.21.15.69:6809/3950661294] conn(0x7fd5b007cad0 0x7fd5b007ef20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.243+0000 7fd0537fe640 1 --2- 172.21.15.69:0/1566520778 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd040074580 0x7fd040076a90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd048034000 tx=0x7fd04802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.243+0000 7fbfadffb640 1 -- 172.21.15.69:0/757339246 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbfb40b4e70 con 0x7fbfb00e73c0 2024-04-25T10:59:24.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.243+0000 7fbfadffb640 1 --2- 172.21.15.69:0/757339246 >> [v2:172.21.15.69:6816/1593617983,v1:172.21.15.69:6817/1593617983] conn(0x7fbf9c07c800 0x7fbf9c07ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.243+0000 7fbfadffb640 1 -- 172.21.15.69:0/757339246 --> [v2:172.21.15.69:6816/1593617983,v1:172.21.15.69:6817/1593617983] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbfb4003e50 con 0x7fbf9c07c800 2024-04-25T10:59:24.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.243+0000 7f4a59ffb640 1 --2- 172.21.15.69:0/3932877831 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4a48074500 0x7f4a48076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.243+0000 7fbfbd23b640 1 --2- 172.21.15.69:0/757339246 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fbf9c0745b0 0x7fbf9c076ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.243+0000 7fbfadffb640 1 -- 172.21.15.69:0/757339246 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fbfb407b700 con 0x7fbfb00e73c0 2024-04-25T10:59:24.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.243+0000 7f4a59ffb640 1 -- 172.21.15.69:0/3932877831 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4a600958b0 con 0x7f4a640cde90 2024-04-25T10:59:24.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.243+0000 7f4a59ffb640 1 --2- 172.21.15.69:0/3932877831 >> [v2:172.21.15.146:6800/2508929872,v1:172.21.15.146:6801/2508929872] conn(0x7f4a4807c750 0x7f4a4807eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.243+0000 7f4a59ffb640 1 -- 172.21.15.69:0/3932877831 --> [v2:172.21.15.146:6800/2508929872,v1:172.21.15.146:6801/2508929872] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a4807f270 con 0x7f4a4807c750 2024-04-25T10:59:24.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.244+0000 7fd050ff9640 1 -- 172.21.15.69:0/1566520778 <== osd.0 v2:172.21.15.69:6824/1876101065 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd04401de50 con 0x7fd04007c850 2024-04-25T10:59:24.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.244+0000 7f827ffff640 1 -- 172.21.15.69:0/1355111125 <== osd.5 v2:172.21.15.146:6809/1310911138 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8288014e50 con 0x7f827807c9f0 2024-04-25T10:59:24.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.244+0000 7fd9457fa640 1 -- 172.21.15.69:0/1979031114 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd938001030 con 0x7fd96014ad60 2024-04-25T10:59:24.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.244+0000 7fbfbca3a640 1 --2- 172.21.15.69:0/757339246 >> [v2:172.21.15.69:6816/1593617983,v1:172.21.15.69:6817/1593617983] conn(0x7fbf9c07c800 0x7fbf9c07ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.244+0000 7fd5beffd640 1 -- 172.21.15.69:0/535421077 <== osd.3 v2:172.21.15.69:6808/3950661294 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd5c0033ec0 con 0x7fd5b007cad0 2024-04-25T10:59:24.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.244+0000 7fbfbd23b640 1 --2- 172.21.15.69:0/757339246 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fbf9c0745b0 0x7fbf9c076ac0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8006fd0 tx=0x7fbfa8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.244+0000 7f4a59ffb640 1 -- 172.21.15.69:0/3932877831 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4a6005c0b0 con 0x7f4a640cde90 2024-04-25T10:59:24.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.244+0000 7fbfbca3a640 1 --2- 172.21.15.69:0/757339246 >> [v2:172.21.15.69:6816/1593617983,v1:172.21.15.69:6817/1593617983] conn(0x7fbf9c07c800 0x7fbf9c07ec50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.245+0000 7eff5e7fc640 1 -- 172.21.15.69:0/2205882014 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7eff680037b0 con 0x7eff6c0aeb80 2024-04-25T10:59:24.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.246+0000 7eff5e7fc640 1 --2- 172.21.15.69:0/2205882014 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7eff500744b0 0x7eff500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.245+0000 7f4a68918640 1 --2- 172.21.15.69:0/3932877831 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4a48074500 0x7f4a48076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.245+0000 7f4a68918640 1 --2- 172.21.15.69:0/3932877831 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4a48074500 0x7f4a48076a10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009560 tx=0x7f4a54009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.245+0000 7fd9477fe640 1 -- 172.21.15.69:0/1979031114 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd954043020 con 0x7fd96014ad60 2024-04-25T10:59:24.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.245+0000 7fd9477fe640 1 --2- 172.21.15.69:0/1979031114 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd94c0745b0 0x7fd94c076ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.245+0000 7fd9477fe640 1 -- 172.21.15.69:0/1979031114 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd9540b57c0 con 0x7fd96014ad60 2024-04-25T10:59:24.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.245+0000 7fd9477fe640 1 --2- 172.21.15.69:0/1979031114 >> [v2:172.21.15.146:6824/3814215935,v1:172.21.15.146:6825/3814215935] conn(0x7fd94c07c820 0x7fd94c07ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.245+0000 7fd9477fe640 1 -- 172.21.15.69:0/1979031114 --> [v2:172.21.15.146:6824/3814215935,v1:172.21.15.146:6825/3814215935] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd954033e50 con 0x7fd94c07c820 2024-04-25T10:59:24.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.245+0000 7fd9477fe640 1 -- 172.21.15.69:0/1979031114 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd9540bc050 con 0x7fd96014ad60 2024-04-25T10:59:24.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.246+0000 7fd95ed9d640 1 --2- 172.21.15.69:0/1979031114 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd94c0745b0 0x7fd94c076ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.246+0000 7eff5e7fc640 1 -- 172.21.15.69:0/2205882014 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7eff680b6820 con 0x7eff6c0aeb80 2024-04-25T10:59:24.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.246+0000 7f4a6991a640 1 --2- 172.21.15.69:0/3932877831 >> [v2:172.21.15.146:6800/2508929872,v1:172.21.15.146:6801/2508929872] conn(0x7f4a4807c750 0x7f4a4807eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.246+0000 7eff5e7fc640 1 --2- 172.21.15.69:0/2205882014 >> [v2:172.21.15.69:6800/1806438563,v1:172.21.15.69:6801/1806438563] conn(0x7eff5007c780 0x7eff5007ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.246+0000 7eff5e7fc640 1 -- 172.21.15.69:0/2205882014 --> [v2:172.21.15.69:6800/1806438563,v1:172.21.15.69:6801/1806438563] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7eff5007f2a0 con 0x7eff5007c780 2024-04-25T10:59:24.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.246+0000 7eff5e7fc640 1 -- 172.21.15.69:0/2205882014 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7eff680b6c10 con 0x7eff6c0aeb80 2024-04-25T10:59:24.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.246+0000 7fd95e59c640 1 --2- 172.21.15.69:0/1979031114 >> [v2:172.21.15.146:6824/3814215935,v1:172.21.15.146:6825/3814215935] conn(0x7fd94c07c820 0x7fd94c07ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.246+0000 7fbfadffb640 1 -- 172.21.15.69:0/757339246 <== osd.2 v2:172.21.15.69:6816/1593617983 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbfb4003e50 con 0x7fbf9c07c800 2024-04-25T10:59:24.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.246+0000 7fd95ed9d640 1 --2- 172.21.15.69:0/1979031114 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd94c0745b0 0x7fd94c076ac0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd950005880 tx=0x7fd950009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.246+0000 7eff726d5640 1 --2- 172.21.15.69:0/2205882014 >> [v2:172.21.15.69:6800/1806438563,v1:172.21.15.69:6801/1806438563] conn(0x7eff5007c780 0x7eff5007ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.246+0000 7eff71ed4640 1 --2- 172.21.15.69:0/2205882014 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7eff500744b0 0x7eff500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.247+0000 7eff726d5640 1 --2- 172.21.15.69:0/2205882014 >> [v2:172.21.15.69:6800/1806438563,v1:172.21.15.69:6801/1806438563] conn(0x7eff5007c780 0x7eff5007ebd0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.246+0000 7f9e7cff9640 1 -- 172.21.15.69:0/48053747 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9e7801c070 con 0x7f9e800b7f70 2024-04-25T10:59:24.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.247+0000 7f9e7cff9640 1 --2- 172.21.15.69:0/48053747 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f9e6c074580 0x7f9e6c076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.247+0000 7f9e7cff9640 1 -- 172.21.15.69:0/48053747 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9e780959a0 con 0x7f9e800b7f70 2024-04-25T10:59:24.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.247+0000 7f9e7ffff640 1 --2- 172.21.15.69:0/48053747 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f9e6c074580 0x7f9e6c076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.247+0000 7fd95e59c640 1 --2- 172.21.15.69:0/1979031114 >> [v2:172.21.15.146:6824/3814215935,v1:172.21.15.146:6825/3814215935] conn(0x7fd94c07c820 0x7fd94c07ec70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.247+0000 7f4a6991a640 1 --2- 172.21.15.69:0/3932877831 >> [v2:172.21.15.146:6800/2508929872,v1:172.21.15.146:6801/2508929872] conn(0x7f4a4807c750 0x7f4a4807eba0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.248+0000 7f9e7cff9640 1 --2- 172.21.15.69:0/48053747 >> [v2:172.21.15.146:6808/634688021,v1:172.21.15.146:6810/634688021] conn(0x7f9e6c07c850 0x7f9e6c07eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.248+0000 7f9e7cff9640 1 -- 172.21.15.69:0/48053747 --> [v2:172.21.15.146:6808/634688021,v1:172.21.15.146:6810/634688021] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9e78018ec0 con 0x7f9e6c07c850 2024-04-25T10:59:24.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.248+0000 7f9e7cff9640 1 -- 172.21.15.69:0/48053747 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9e78095d90 con 0x7f9e800b7f70 2024-04-25T10:59:24.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.248+0000 7eff71ed4640 1 --2- 172.21.15.69:0/2205882014 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7eff500744b0 0x7eff500769c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7eff60006fd0 tx=0x7eff60008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.248+0000 7f9e7f7fe640 1 --2- 172.21.15.69:0/48053747 >> [v2:172.21.15.146:6808/634688021,v1:172.21.15.146:6810/634688021] conn(0x7f9e6c07c850 0x7f9e6c07eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.248+0000 7f9e7ffff640 1 --2- 172.21.15.69:0/48053747 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f9e6c074580 0x7f9e6c076a90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9e700010e0 tx=0x7f9e70009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.248+0000 7eff5e7fc640 1 -- 172.21.15.69:0/2205882014 <== osd.1 v2:172.21.15.69:6800/1806438563 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7eff5007f2a0 con 0x7eff5007c780 2024-04-25T10:59:24.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.248+0000 7fd9477fe640 1 -- 172.21.15.69:0/1979031114 <== osd.7 v2:172.21.15.146:6824/3814215935 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd954033e50 con 0x7fd94c07c820 2024-04-25T10:59:24.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.248+0000 7f9e7f7fe640 1 --2- 172.21.15.69:0/48053747 >> [v2:172.21.15.146:6808/634688021,v1:172.21.15.146:6810/634688021] conn(0x7f9e6c07c850 0x7f9e6c07eca0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.248+0000 7f4a59ffb640 1 -- 172.21.15.69:0/3932877831 <== osd.4 v2:172.21.15.146:6800/2508929872 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4a4807f270 con 0x7f4a4807c750 2024-04-25T10:59:24.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.251+0000 7f9e7cff9640 1 -- 172.21.15.69:0/48053747 <== osd.6 v2:172.21.15.146:6808/634688021 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9e78018ec0 con 0x7f9e6c07c850 2024-04-25T10:59:24.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.276+0000 7fd05a6ca640 1 -- 172.21.15.69:0/1566520778 --> [v2:172.21.15.69:6824/1876101065,v1:172.21.15.69:6825/1876101065] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd034002dd0 con 0x7fd04007c850 2024-04-25T10:59:24.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.276+0000 7fd5d4d14640 1 -- 172.21.15.69:0/535421077 --> [v2:172.21.15.69:6808/3950661294,v1:172.21.15.69:6809/3950661294] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd5d0163310 con 0x7fd5b007cad0 2024-04-25T10:59:24.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.276+0000 7fd050ff9640 1 -- 172.21.15.69:0/1566520778 <== osd.0 v2:172.21.15.69:6824/1876101065 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd034002dd0 con 0x7fd04007c850 2024-04-25T10:59:24.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.277+0000 7f8299ef9640 1 -- 172.21.15.69:0/1355111125 --> [v2:172.21.15.146:6809/1310911138,v1:172.21.15.146:6811/1310911138] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f828c005570 con 0x7f827807c9f0 2024-04-25T10:59:24.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.277+0000 7fd05a6ca640 1 -- 172.21.15.69:0/1566520778 >> [v2:172.21.15.69:6824/1876101065,v1:172.21.15.69:6825/1876101065] conn(0x7fd04007c850 msgr2=0x7fd04007eca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.277+0000 7fd05a6ca640 1 --2- 172.21.15.69:0/1566520778 >> [v2:172.21.15.69:6824/1876101065,v1:172.21.15.69:6825/1876101065] conn(0x7fd04007c850 0x7fd04007eca0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.277+0000 7eff74138640 1 -- 172.21.15.69:0/2205882014 --> [v2:172.21.15.69:6800/1806438563,v1:172.21.15.69:6801/1806438563] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7eff6c1567d0 con 0x7eff5007c780 2024-04-25T10:59:24.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.277+0000 7fd5beffd640 1 -- 172.21.15.69:0/535421077 <== osd.3 v2:172.21.15.69:6808/3950661294 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd5d0163310 con 0x7fd5b007cad0 2024-04-25T10:59:24.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.277+0000 7fd05a6ca640 1 -- 172.21.15.69:0/1566520778 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd040074580 msgr2=0x7fd040076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.277+0000 7fd05a6ca640 1 --2- 172.21.15.69:0/1566520778 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd040074580 0x7fd040076a90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd048034000 tx=0x7fd04802e000 comp rx=0 tx=0).stop 2024-04-25T10:59:24.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.277+0000 7fd05a6ca640 1 -- 172.21.15.69:0/1566520778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd05406e030 msgr2=0x7fd054073100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.277+0000 7fd05a6ca640 1 --2- 172.21.15.69:0/1566520778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd05406e030 0x7fd054073100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd04400d650 tx=0x7fd04400db10 comp rx=0 tx=0).stop 2024-04-25T10:59:24.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.277+0000 7fd058c67640 1 -- 172.21.15.69:0/1566520778 reap_dead start 2024-04-25T10:59:24.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.277+0000 7fd05a6ca640 1 -- 172.21.15.69:0/1566520778 shutdown_connections 2024-04-25T10:59:24.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.277+0000 7fd05a6ca640 1 -- 172.21.15.69:0/1566520778 >> 172.21.15.69:0/1566520778 conn(0x7fd0540d27c0 msgr2=0x7fd054163360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:24.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.277+0000 7fd5d4d14640 1 -- 172.21.15.69:0/535421077 >> [v2:172.21.15.69:6808/3950661294,v1:172.21.15.69:6809/3950661294] conn(0x7fd5b007cad0 msgr2=0x7fd5b007ef20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.278+0000 7eff5e7fc640 1 -- 172.21.15.69:0/2205882014 <== osd.1 v2:172.21.15.69:6800/1806438563 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7eff6c1567d0 con 0x7eff5007c780 2024-04-25T10:59:24.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.277+0000 7fd5d4d14640 1 --2- 172.21.15.69:0/535421077 >> [v2:172.21.15.69:6808/3950661294,v1:172.21.15.69:6809/3950661294] conn(0x7fd5b007cad0 0x7fd5b007ef20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.278+0000 7fd05a6ca640 1 -- 172.21.15.69:0/1566520778 shutdown_connections 2024-04-25T10:59:24.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.278+0000 7fd5d4d14640 1 -- 172.21.15.69:0/535421077 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd5b00748e0 msgr2=0x7fd5b0076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.278+0000 7fd5d4d14640 1 --2- 172.21.15.69:0/535421077 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd5b00748e0 0x7fd5b0076df0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8006a90 tx=0x7fd5c8008040 comp rx=0 tx=0).stop 2024-04-25T10:59:24.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.278+0000 7fd5d4d14640 1 -- 172.21.15.69:0/535421077 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5d01681d0 msgr2=0x7fd5d015e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.278+0000 7fd5d4d14640 1 --2- 172.21.15.69:0/535421077 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5d01681d0 0x7fd5d015e900 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0030000 tx=0x7fd5c0004a40 comp rx=0 tx=0).stop 2024-04-25T10:59:24.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.278+0000 7fd05a6ca640 1 -- 172.21.15.69:0/1566520778 wait complete. 2024-04-25T10:59:24.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.278+0000 7f827ffff640 1 -- 172.21.15.69:0/1355111125 <== osd.5 v2:172.21.15.146:6809/1310911138 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f828c005570 con 0x7f827807c9f0 2024-04-25T10:59:24.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.278+0000 7fd5ced9d640 1 -- 172.21.15.69:0/535421077 reap_dead start 2024-04-25T10:59:24.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.278+0000 7f8299ef9640 1 -- 172.21.15.69:0/1355111125 >> [v2:172.21.15.146:6809/1310911138,v1:172.21.15.146:6811/1310911138] conn(0x7f827807c9f0 msgr2=0x7f827807ee40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.278+0000 7eff74138640 1 -- 172.21.15.69:0/2205882014 >> [v2:172.21.15.69:6800/1806438563,v1:172.21.15.69:6801/1806438563] conn(0x7eff5007c780 msgr2=0x7eff5007ebd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.278+0000 7eff74138640 1 --2- 172.21.15.69:0/2205882014 >> [v2:172.21.15.69:6800/1806438563,v1:172.21.15.69:6801/1806438563] conn(0x7eff5007c780 0x7eff5007ebd0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.278+0000 7f8299ef9640 1 --2- 172.21.15.69:0/1355111125 >> [v2:172.21.15.146:6809/1310911138,v1:172.21.15.146:6811/1310911138] conn(0x7f827807c9f0 0x7f827807ee40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.279+0000 7eff74138640 1 -- 172.21.15.69:0/2205882014 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7eff500744b0 msgr2=0x7eff500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.279+0000 7eff74138640 1 --2- 172.21.15.69:0/2205882014 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7eff500744b0 0x7eff500769c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7eff60006fd0 tx=0x7eff60008040 comp rx=0 tx=0).stop 2024-04-25T10:59:24.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.279+0000 7eff74138640 1 -- 172.21.15.69:0/2205882014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff6c0aeb80 msgr2=0x7eff6c15fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.279+0000 7eff74138640 1 --2- 172.21.15.69:0/2205882014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff6c0aeb80 0x7eff6c15fc10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7eff68000c00 tx=0x7eff680046c0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.279+0000 7f8299ef9640 1 -- 172.21.15.69:0/1355111125 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f8278074720 msgr2=0x7f8278076c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.279+0000 7f8299ef9640 1 --2- 172.21.15.69:0/1355111125 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f8278074720 0x7f8278076c30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8280001950 tx=0x7f82800018a0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.279+0000 7f8299ef9640 1 -- 172.21.15.69:0/1355111125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f828c047db0 msgr2=0x7f828c0ea3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.279+0000 7f8299ef9640 1 --2- 172.21.15.69:0/1355111125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f828c047db0 0x7f828c0ea3f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f828800c480 tx=0x7f828800c940 comp rx=0 tx=0).stop 2024-04-25T10:59:24.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.279+0000 7f8293fff640 1 -- 172.21.15.69:0/1355111125 reap_dead start 2024-04-25T10:59:24.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.279+0000 7eff726d5640 1 -- 172.21.15.69:0/2205882014 reap_dead start 2024-04-25T10:59:24.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.279+0000 7fd5d4d14640 1 -- 172.21.15.69:0/535421077 shutdown_connections 2024-04-25T10:59:24.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.279+0000 7fd5d4d14640 1 -- 172.21.15.69:0/535421077 >> 172.21.15.69:0/535421077 conn(0x7fd5d00d1ae0 msgr2=0x7fd5d00d06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:24.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.279+0000 7fd5d4d14640 1 -- 172.21.15.69:0/535421077 shutdown_connections 2024-04-25T10:59:24.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.279+0000 7fd5d4d14640 1 -- 172.21.15.69:0/535421077 wait complete. 2024-04-25T10:59:24.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.279+0000 7f8299ef9640 1 -- 172.21.15.69:0/1355111125 shutdown_connections 2024-04-25T10:59:24.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.279+0000 7f8299ef9640 1 -- 172.21.15.69:0/1355111125 >> 172.21.15.69:0/1355111125 conn(0x7f828c054cf0 msgr2=0x7f828c005ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:24.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.279+0000 7eff74138640 1 -- 172.21.15.69:0/2205882014 shutdown_connections 2024-04-25T10:59:24.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.279+0000 7eff74138640 1 -- 172.21.15.69:0/2205882014 >> 172.21.15.69:0/2205882014 conn(0x7eff6c0d1ed0 msgr2=0x7eff6c073cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:24.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.279+0000 7f8299ef9640 1 -- 172.21.15.69:0/1355111125 shutdown_connections 2024-04-25T10:59:24.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.279+0000 7f8299ef9640 1 -- 172.21.15.69:0/1355111125 wait complete. 2024-04-25T10:59:24.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.280+0000 7eff74138640 1 -- 172.21.15.69:0/2205882014 shutdown_connections 2024-04-25T10:59:24.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.280+0000 7eff74138640 1 -- 172.21.15.69:0/2205882014 wait complete. 2024-04-25T10:59:24.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.280+0000 7fbfbe23d640 1 -- 172.21.15.69:0/757339246 --> [v2:172.21.15.69:6816/1593617983,v1:172.21.15.69:6817/1593617983] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbf98002dd0 con 0x7fbf9c07c800 2024-04-25T10:59:24.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.280+0000 7f4a6b37d640 1 -- 172.21.15.69:0/3932877831 --> [v2:172.21.15.146:6800/2508929872,v1:172.21.15.146:6801/2508929872] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4a64050740 con 0x7f4a4807c750 2024-04-25T10:59:24.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.280+0000 7fd9457fa640 1 -- 172.21.15.69:0/1979031114 --> [v2:172.21.15.146:6824/3814215935,v1:172.21.15.146:6825/3814215935] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd938002dd0 con 0x7fd94c07c820 2024-04-25T10:59:24.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.281+0000 7fbfadffb640 1 -- 172.21.15.69:0/757339246 <== osd.2 v2:172.21.15.69:6816/1593617983 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbf98002dd0 con 0x7fbf9c07c800 2024-04-25T10:59:24.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.281+0000 7f4a59ffb640 1 -- 172.21.15.69:0/3932877831 <== osd.4 v2:172.21.15.146:6800/2508929872 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4a64050740 con 0x7f4a4807c750 2024-04-25T10:59:24.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.281+0000 7fd9477fe640 1 -- 172.21.15.69:0/1979031114 <== osd.7 v2:172.21.15.146:6824/3814215935 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd938002dd0 con 0x7fd94c07c820 2024-04-25T10:59:24.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.281+0000 7fbfbe23d640 1 -- 172.21.15.69:0/757339246 >> [v2:172.21.15.69:6816/1593617983,v1:172.21.15.69:6817/1593617983] conn(0x7fbf9c07c800 msgr2=0x7fbf9c07ec50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.282+0000 7fbfbe23d640 1 --2- 172.21.15.69:0/757339246 >> [v2:172.21.15.69:6816/1593617983,v1:172.21.15.69:6817/1593617983] conn(0x7fbf9c07c800 0x7fbf9c07ec50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.282+0000 7fbfbe23d640 1 -- 172.21.15.69:0/757339246 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fbf9c0745b0 msgr2=0x7fbf9c076ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.282+0000 7fbfbe23d640 1 --2- 172.21.15.69:0/757339246 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fbf9c0745b0 0x7fbf9c076ac0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8006fd0 tx=0x7fbfa8008040 comp rx=0 tx=0).stop 2024-04-25T10:59:24.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.282+0000 7fbfbe23d640 1 -- 172.21.15.69:0/757339246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbfb00e73c0 msgr2=0x7fbfb00dfc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.282+0000 7fbfbe23d640 1 --2- 172.21.15.69:0/757339246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbfb00e73c0 0x7fbfb00dfc60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4031000 tx=0x7fbfb40046c0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.282+0000 7f4a6b37d640 1 -- 172.21.15.69:0/3932877831 >> [v2:172.21.15.146:6800/2508929872,v1:172.21.15.146:6801/2508929872] conn(0x7f4a4807c750 msgr2=0x7f4a4807eba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.282+0000 7f4a6b37d640 1 --2- 172.21.15.69:0/3932877831 >> [v2:172.21.15.146:6800/2508929872,v1:172.21.15.146:6801/2508929872] conn(0x7f4a4807c750 0x7f4a4807eba0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.282+0000 7fbfbda3c640 1 -- 172.21.15.69:0/757339246 reap_dead start 2024-04-25T10:59:24.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.282+0000 7f4a6b37d640 1 -- 172.21.15.69:0/3932877831 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4a48074500 msgr2=0x7f4a48076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.282+0000 7fbfbe23d640 1 -- 172.21.15.69:0/757339246 shutdown_connections 2024-04-25T10:59:24.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.282+0000 7f4a6b37d640 1 --2- 172.21.15.69:0/3932877831 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4a48074500 0x7f4a48076a10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009560 tx=0x7f4a54009190 comp rx=0 tx=0).stop 2024-04-25T10:59:24.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.282+0000 7fbfbe23d640 1 -- 172.21.15.69:0/757339246 >> 172.21.15.69:0/757339246 conn(0x7fbfb003ac70 msgr2=0x7fbfb00e2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:24.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.282+0000 7f4a6b37d640 1 -- 172.21.15.69:0/3932877831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a640cde90 msgr2=0x7f4a640c7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.282+0000 7f4a6b37d640 1 --2- 172.21.15.69:0/3932877831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a640cde90 0x7f4a640c7ee0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000b520 tx=0x7f4a6000b9e0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.282+0000 7f4a6991a640 1 -- 172.21.15.69:0/3932877831 reap_dead start 2024-04-25T10:59:24.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.282+0000 7fbfbe23d640 1 -- 172.21.15.69:0/757339246 shutdown_connections 2024-04-25T10:59:24.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.282+0000 7fbfbe23d640 1 -- 172.21.15.69:0/757339246 wait complete. 2024-04-25T10:59:24.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.283+0000 7f4a6b37d640 1 -- 172.21.15.69:0/3932877831 shutdown_connections 2024-04-25T10:59:24.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.283+0000 7f4a6b37d640 1 -- 172.21.15.69:0/3932877831 >> 172.21.15.69:0/3932877831 conn(0x7f4a64073050 msgr2=0x7f4a640b2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:24.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.284+0000 7fd9457fa640 1 -- 172.21.15.69:0/1979031114 >> [v2:172.21.15.146:6824/3814215935,v1:172.21.15.146:6825/3814215935] conn(0x7fd94c07c820 msgr2=0x7fd94c07ec70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.284+0000 7fd9457fa640 1 --2- 172.21.15.69:0/1979031114 >> [v2:172.21.15.146:6824/3814215935,v1:172.21.15.146:6825/3814215935] conn(0x7fd94c07c820 0x7fd94c07ec70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.284+0000 7f4a6b37d640 1 -- 172.21.15.69:0/3932877831 shutdown_connections 2024-04-25T10:59:24.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.284+0000 7f4a6b37d640 1 -- 172.21.15.69:0/3932877831 wait complete. 2024-04-25T10:59:24.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.284+0000 7fd9457fa640 1 -- 172.21.15.69:0/1979031114 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd94c0745b0 msgr2=0x7fd94c076ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.284+0000 7fd9457fa640 1 --2- 172.21.15.69:0/1979031114 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd94c0745b0 0x7fd94c076ac0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd950005880 tx=0x7fd950009040 comp rx=0 tx=0).stop 2024-04-25T10:59:24.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.284+0000 7f9e86746640 1 -- 172.21.15.69:0/48053747 --> [v2:172.21.15.146:6808/634688021,v1:172.21.15.146:6810/634688021] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9e800a0590 con 0x7f9e6c07c850 2024-04-25T10:59:24.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.284+0000 7fd9457fa640 1 -- 172.21.15.69:0/1979031114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd96014ad60 msgr2=0x7fd9600b08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.284+0000 7fd9457fa640 1 --2- 172.21.15.69:0/1979031114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd96014ad60 0x7fd9600b08e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd954031000 tx=0x7fd9540046c0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.284+0000 7fd95f59e640 1 -- 172.21.15.69:0/1979031114 reap_dead start 2024-04-25T10:59:24.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.284+0000 7fd9457fa640 1 -- 172.21.15.69:0/1979031114 shutdown_connections 2024-04-25T10:59:24.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.284+0000 7fd9457fa640 1 -- 172.21.15.69:0/1979031114 >> 172.21.15.69:0/1979031114 conn(0x7fd9600d1ae0 msgr2=0x7fd9600b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:24.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.284+0000 7f9e7cff9640 1 -- 172.21.15.69:0/48053747 <== osd.6 v2:172.21.15.146:6808/634688021 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9e800a0590 con 0x7f9e6c07c850 2024-04-25T10:59:24.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.285+0000 7fd9457fa640 1 -- 172.21.15.69:0/1979031114 shutdown_connections 2024-04-25T10:59:24.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.285+0000 7fd9457fa640 1 -- 172.21.15.69:0/1979031114 wait complete. 2024-04-25T10:59:24.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.285+0000 7f9e667fc640 1 -- 172.21.15.69:0/48053747 >> [v2:172.21.15.146:6808/634688021,v1:172.21.15.146:6810/634688021] conn(0x7f9e6c07c850 msgr2=0x7f9e6c07eca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.285+0000 7f9e667fc640 1 --2- 172.21.15.69:0/48053747 >> [v2:172.21.15.146:6808/634688021,v1:172.21.15.146:6810/634688021] conn(0x7f9e6c07c850 0x7f9e6c07eca0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.286+0000 7f9e667fc640 1 -- 172.21.15.69:0/48053747 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f9e6c074580 msgr2=0x7f9e6c076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.286+0000 7f9e667fc640 1 --2- 172.21.15.69:0/48053747 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f9e6c074580 0x7f9e6c076a90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9e700010e0 tx=0x7f9e70009040 comp rx=0 tx=0).stop 2024-04-25T10:59:24.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.286+0000 7f9e667fc640 1 -- 172.21.15.69:0/48053747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e800b7f70 msgr2=0x7f9e8009d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.286+0000 7f9e667fc640 1 --2- 172.21.15.69:0/48053747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e800b7f70 0x7f9e8009d7f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9e7800b990 tx=0x7f9e7800be50 comp rx=0 tx=0).stop 2024-04-25T10:59:24.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.286+0000 7f9e84ce3640 1 -- 172.21.15.69:0/48053747 reap_dead start 2024-04-25T10:59:24.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.286+0000 7f9e667fc640 1 -- 172.21.15.69:0/48053747 shutdown_connections 2024-04-25T10:59:24.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.287+0000 7f9e667fc640 1 -- 172.21.15.69:0/48053747 >> 172.21.15.69:0/48053747 conn(0x7f9e800b5640 msgr2=0x7f9e800d2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:24.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.287+0000 7f9e667fc640 1 -- 172.21.15.69:0/48053747 shutdown_connections 2024-04-25T10:59:24.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.287+0000 7f9e667fc640 1 -- 172.21.15.69:0/48053747 wait complete. 2024-04-25T10:59:24.344 INFO:teuthology.orchestra.run.smithi069.stdout:55834574855 2024-04-25T10:59:24.344 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T10:59:24.345 INFO:teuthology.orchestra.run.smithi069.stdout:55834574856 2024-04-25T10:59:24.345 INFO:teuthology.orchestra.run.smithi069.stdout:55834574856 2024-04-25T10:59:24.346 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T10:59:24.346 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T10:59:24.347 INFO:teuthology.orchestra.run.smithi069.stdout:55834574855 2024-04-25T10:59:24.347 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T10:59:24.348 INFO:teuthology.orchestra.run.smithi069.stdout:55834574856 2024-04-25T10:59:24.348 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T10:59:24.349 INFO:teuthology.orchestra.run.smithi069.stdout:55834574855 2024-04-25T10:59:24.350 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T10:59:24.350 INFO:teuthology.orchestra.run.smithi069.stdout:55834574855 2024-04-25T10:59:24.351 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T10:59:24.351 INFO:teuthology.orchestra.run.smithi069.stdout:55834574856 2024-04-25T10:59:24.352 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T10:59:24.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.588+0000 7fc46199d640 1 Processor -- start 2024-04-25T10:59:24.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.588+0000 7fc46199d640 1 -- start start 2024-04-25T10:59:24.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.590+0000 7fc46199d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc454104f00 0x7fc454052f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.590+0000 7fc46199d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc45404fb30 0x7fc45404ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.590+0000 7fc46199d640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fc454006090 0x7fc4540064a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.590+0000 7fc46199d640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fc454050480 con 0x7fc454006090 2024-04-25T10:59:24.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.590+0000 7fc46199d640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fc454053670 con 0x7fc45404fb30 2024-04-25T10:59:24.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.590+0000 7fc46199d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc454053950 con 0x7fc454104f00 2024-04-25T10:59:24.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.590+0000 7fc46119c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fc454006090 0x7fc4540064a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.590+0000 7fc46119c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fc454006090 0x7fc4540064a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41380/0 (socket says 172.21.15.69:41380) 2024-04-25T10:59:24.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.590+0000 7fc46119c640 1 -- 172.21.15.69:0/167293318 learned_addr learned my addr 172.21.15.69:0/167293318 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:24.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.591+0000 7fc46119c640 1 -- 172.21.15.69:0/167293318 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc454104f00 msgr2=0x7fc454052f90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:59:24.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.591+0000 7fc46119c640 1 --2- 172.21.15.69:0/167293318 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc454104f00 0x7fc454052f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.591+0000 7fc46119c640 1 -- 172.21.15.69:0/167293318 >> v2:172.21.15.146:3300/0 conn(0x7fc45404fb30 msgr2=0x7fc45404ff40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:59:24.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.591+0000 7fc46119c640 1 --2- 172.21.15.69:0/167293318 >> v2:172.21.15.146:3300/0 conn(0x7fc45404fb30 0x7fc45404ff40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.592+0000 7f7975dee640 1 Processor -- start 2024-04-25T10:59:24.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.592+0000 7f7975dee640 1 -- start start 2024-04-25T10:59:24.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.591+0000 7fc46119c640 1 -- 172.21.15.69:0/167293318 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc454053c30 con 0x7fc454006090 2024-04-25T10:59:24.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.593+0000 7fc46119c640 1 --2- 172.21.15.69:0/167293318 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc454006090 0x7fc4540064a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc45c06b8e0 tx=0x7fc45c0695d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27377d7cbb42ef5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.593+0000 7f6cf5320640 1 Processor -- start 2024-04-25T10:59:24.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.593+0000 7f6cf5320640 1 -- start start 2024-04-25T10:59:24.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.593+0000 7f71832d6640 1 Processor -- start 2024-04-25T10:59:24.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.592+0000 7faefdb55640 1 Processor -- start 2024-04-25T10:59:24.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.593+0000 7f71832d6640 1 -- start start 2024-04-25T10:59:24.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.592+0000 7faefdb55640 1 -- start start 2024-04-25T10:59:24.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.594+0000 7fc45b7fe640 1 -- 172.21.15.69:0/167293318 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c0a4070 con 0x7fc454006090 2024-04-25T10:59:24.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.594+0000 7fc45b7fe640 1 -- 172.21.15.69:0/167293318 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc45c09f600 con 0x7fc454006090 2024-04-25T10:59:24.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.594+0000 7fc45b7fe640 1 -- 172.21.15.69:0/167293318 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c09fa70 con 0x7fc454006090 2024-04-25T10:59:24.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.594+0000 7f99c6dee640 1 Processor -- start 2024-04-25T10:59:24.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.594+0000 7f99c6dee640 1 -- start start 2024-04-25T10:59:24.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.594+0000 7f7975dee640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f797014dd70 0x7f7970070f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.594+0000 7f7975dee640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f797006d5d0 0x7f797006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.594+0000 7f7975dee640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f797006df20 con 0x7f7970168250 2024-04-25T10:59:24.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.594+0000 7f7975dee640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f79700b4970 con 0x7f797006d5d0 2024-04-25T10:59:24.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.594+0000 7f7975dee640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f79700b57d0 con 0x7f797014dd70 2024-04-25T10:59:24.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.594+0000 7f80bf59e640 1 Processor -- start 2024-04-25T10:59:24.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.595+0000 7faefdb55640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faef0034060 0x7faef00e7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.595+0000 7f796effd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f797006d5d0 0x7f797006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.595+0000 7faefdb55640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7faef0048300 0x7faef00065e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.595+0000 7f796effd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f797006d5d0 0x7f797006d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.69:51870/0 (socket says 172.21.15.69:51870) 2024-04-25T10:59:24.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.595+0000 7faefdb55640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faef0006130 con 0x7faef0047db0 2024-04-25T10:59:24.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.595+0000 7faefdb55640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7faef004f690 con 0x7faef0048300 2024-04-25T10:59:24.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.595+0000 7faefdb55640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7faef004ffa0 con 0x7faef0034060 2024-04-25T10:59:24.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.595+0000 7f796effd640 1 -- 172.21.15.69:0/3673058945 learned_addr learned my addr 172.21.15.69:0/3673058945 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:24.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.595+0000 7f796f7fe640 1 --2- 172.21.15.69:0/3673058945 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f797014dd70 0x7f7970070f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.595+0000 7faef7fff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faef0034060 0x7faef00e7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.595+0000 7faef7fff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faef0034060 0x7faef00e7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.69:58262/0 (socket says 172.21.15.69:58262) 2024-04-25T10:59:24.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.595+0000 7f796effd640 1 -- 172.21.15.69:0/3673058945 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f797014dd70 msgr2=0x7f7970070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:24.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.595+0000 7faef7fff640 1 -- 172.21.15.69:0/1344350051 learned_addr learned my addr 172.21.15.69:0/1344350051 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:24.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.595+0000 7faefd354640 1 --2- 172.21.15.69:0/1344350051 >> v2:172.21.15.146:3300/0 conn(0x7faef0048300 0x7faef00065e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.595+0000 7f6cf5320640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f6ce80340c0 0x7f6ce8006aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.595+0000 7f796e7fc640 1 -- 172.21.15.69:0/3673058945 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1311887955 0 0) 0x7f7968001490 con 0x7f7970168250 2024-04-25T10:59:24.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.595+0000 7f6cf5320640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6ce80dfd90 0x7f6ce80e01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.595+0000 7f796effd640 1 --2- 172.21.15.69:0/3673058945 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f797014dd70 0x7f7970070f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.595+0000 7f796effd640 1 -- 172.21.15.69:0/3673058945 >> v1:172.21.15.69:6789/0 conn(0x7f7970168250 legacy=0x7f7970168660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.595+0000 7f796effd640 1 -- 172.21.15.69:0/3673058945 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f797014e180 con 0x7f797006d5d0 2024-04-25T10:59:24.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.595+0000 7f6cf5320640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ce80e06e0 0x7f6ce80e3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.595+0000 7f80bf59e640 1 -- start start 2024-04-25T10:59:24.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.595+0000 7f6ceffff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f6ce80340c0 0x7f6ce8006aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.595+0000 7f6cf5320640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f6ce80344d0 con 0x7f6ce80340c0 2024-04-25T10:59:24.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.595+0000 7f6ceffff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f6ce80340c0 0x7f6ce8006aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41394/0 (socket says 172.21.15.69:41394) 2024-04-25T10:59:24.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.595+0000 7f6ceffff640 1 -- 172.21.15.69:0/3377156737 learned_addr learned my addr 172.21.15.69:0/3377156737 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:24.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.595+0000 7f6cf5320640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f6ce80071d0 con 0x7f6ce80dfd90 2024-04-25T10:59:24.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.596+0000 7f6cf4b1f640 1 --2- 172.21.15.69:0/3377156737 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ce80e06e0 0x7f6ce80e3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.596+0000 7f6cf5320640 1 -- 172.21.15.69:0/3377156737 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6ce80074b0 con 0x7f6ce80e06e0 2024-04-25T10:59:24.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.596+0000 7f6cef7fe640 1 --2- 172.21.15.69:0/3377156737 >> v2:172.21.15.146:3300/0 conn(0x7f6ce80dfd90 0x7f6ce80e01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.596+0000 7fc46199d640 1 -- 172.21.15.69:0/167293318 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc454006090 msgr2=0x7fc4540064a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.596+0000 7fc46199d640 1 --2- 172.21.15.69:0/167293318 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc454006090 0x7fc4540064a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc45c06b8e0 tx=0x7fc45c0695d0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.596+0000 7fc46199d640 1 -- 172.21.15.69:0/167293318 shutdown_connections 2024-04-25T10:59:24.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.596+0000 7fc46199d640 1 --2- 172.21.15.69:0/167293318 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc454006090 0x7fc4540064a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.596+0000 7fc46199d640 1 --2- 172.21.15.69:0/167293318 >> v2:172.21.15.146:3300/0 conn(0x7fc45404fb30 0x7fc45404ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.596+0000 7fc46199d640 1 --2- 172.21.15.69:0/167293318 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc454104f00 0x7fc454052f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.596+0000 7fc46199d640 1 -- 172.21.15.69:0/167293318 >> 172.21.15.69:0/167293318 conn(0x7fc454054cf0 msgr2=0x7fc454055100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:24.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.597+0000 7f99c6dee640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f99b8050e00 0x7f99b8051210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.597+0000 7f99c6dee640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99b8039ec0 0x7f99b803a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.596+0000 7fc46199d640 1 -- 172.21.15.69:0/167293318 shutdown_connections 2024-04-25T10:59:24.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.596+0000 7f6cef7fe640 1 -- 172.21.15.69:0/3377156737 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ce80e06e0 msgr2=0x7f6ce80e3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:24.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.596+0000 7f6cef7fe640 1 --2- 172.21.15.69:0/3377156737 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ce80e06e0 0x7f6ce80e3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.596+0000 7f6cef7fe640 1 -- 172.21.15.69:0/3377156737 >> v2:172.21.15.69:3300/0 conn(0x7f6ce80340c0 msgr2=0x7f6ce8006aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:24.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.597+0000 7f6cef7fe640 1 --2- 172.21.15.69:0/3377156737 >> v2:172.21.15.69:3300/0 conn(0x7f6ce80340c0 0x7f6ce8006aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.597+0000 7f6cef7fe640 1 -- 172.21.15.69:0/3377156737 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce8007790 con 0x7f6ce80dfd90 2024-04-25T10:59:24.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.597+0000 7f6ceffff640 1 --2- 172.21.15.69:0/3377156737 >> v2:172.21.15.69:3300/0 conn(0x7f6ce80340c0 0x7f6ce8006aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:24.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.597+0000 7f6cf4b1f640 1 --2- 172.21.15.69:0/3377156737 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ce80e06e0 0x7f6ce80e3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:24.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.596+0000 7f71832d6640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7174056440 0x7f71740e2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.596+0000 7f71832d6640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f71740e5b10 0x7f7174009f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.596+0000 7f71832d6640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f717400a8c0 con 0x7f717400a4b0 2024-04-25T10:59:24.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.596+0000 7f71832d6640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f7174054a40 con 0x7f7174056440 2024-04-25T10:59:24.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.596+0000 7f71832d6640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f7174051d80 con 0x7f71740e5b10 2024-04-25T10:59:24.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.597+0000 7f99c6dee640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99b8051750 con 0x7f99b8054520 2024-04-25T10:59:24.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.597+0000 7f99c6dee640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f99b8008ec0 con 0x7f99b8050e00 2024-04-25T10:59:24.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.597+0000 7f99c6dee640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f99b8039a30 con 0x7f99b8039ec0 2024-04-25T10:59:24.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.597+0000 7f99c4b8a640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f99b8054520 0x7f99b80087b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.69:50392/0 (socket says 172.21.15.69:50392) 2024-04-25T10:59:24.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.597+0000 7f7181ad3640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f71740e5b10 0x7f7174009f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.597+0000 7f7182ad5640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f717400a4b0 0x7f71740e4060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:57214/0 (socket says 172.21.15.69:57214) 2024-04-25T10:59:24.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.597+0000 7f7182ad5640 1 -- 172.21.15.69:0/451654147 learned_addr learned my addr 172.21.15.69:0/451654147 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:24.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.597+0000 7f71822d4640 1 --2- 172.21.15.69:0/451654147 >> v2:172.21.15.69:3300/0 conn(0x7f7174056440 0x7f71740e2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.597+0000 7f80bf59e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80c00b46d0 0x7f80c01646b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.597+0000 7f80bf59e640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f80c00d4e90 0x7f80c00d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.597+0000 7f80bf59e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80c00d1ee0 con 0x7f80c00b75b0 2024-04-25T10:59:24.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.597+0000 7f80bf59e640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f80c00d0180 con 0x7f80c00d4e90 2024-04-25T10:59:24.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.597+0000 7f80bf59e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f80c00d21c0 con 0x7f80c00b46d0 2024-04-25T10:59:24.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.596+0000 7fc46199d640 1 -- 172.21.15.69:0/167293318 wait complete. 2024-04-25T10:59:24.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.597+0000 7f99c4b8a640 1 -- 172.21.15.69:0/4262417093 learned_addr learned my addr 172.21.15.69:0/4262417093 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:24.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.597+0000 7f80bed9d640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f80c00b75b0 0x7f80c00b79c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.69:50404/0 (socket says 172.21.15.69:50404) 2024-04-25T10:59:24.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.597+0000 7f80bed9d640 1 -- 172.21.15.69:0/2945299897 learned_addr learned my addr 172.21.15.69:0/2945299897 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:24.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.596+0000 7faef77fe640 1 -- 172.21.15.69:0/1344350051 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4116446895 0 0) 0x7faef0006130 con 0x7faef0047db0 2024-04-25T10:59:24.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.596+0000 7faef77fe640 1 -- 172.21.15.69:0/1344350051 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faee8003520 con 0x7faef0047db0 2024-04-25T10:59:24.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.596+0000 7faefd354640 1 -- 172.21.15.69:0/1344350051 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faef0034060 msgr2=0x7faef00e7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:24.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.596+0000 7faefd354640 1 --2- 172.21.15.69:0/1344350051 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faef0034060 0x7faef00e7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.596+0000 7faefd354640 1 -- 172.21.15.69:0/1344350051 >> v1:172.21.15.69:6789/0 conn(0x7faef0047db0 legacy=0x7faef0033950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.596+0000 7faefd354640 1 -- 172.21.15.69:0/1344350051 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef0050280 con 0x7faef0048300 2024-04-25T10:59:24.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.597+0000 7f80bdd9b640 1 --2- 172.21.15.69:0/2945299897 >> v2:172.21.15.146:3300/0 conn(0x7f80c00d4e90 0x7f80c00d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.597+0000 7f99bffff640 1 --2- 172.21.15.69:0/4262417093 >> v2:172.21.15.146:3300/0 conn(0x7f99b8050e00 0x7f99b8051210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.597+0000 7faef7fff640 1 --2- 172.21.15.69:0/1344350051 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faef0034060 0x7faef00e7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:59:24.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.597+0000 7f99c538b640 1 --2- 172.21.15.69:0/4262417093 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99b8039ec0 0x7f99b803a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.598+0000 7faefd354640 1 --2- 172.21.15.69:0/1344350051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faef0048300 0x7faef00065e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faeec0097a0 tx=0x7faeec02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8a9d2e4c804efce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.598+0000 7f99bf7fe640 1 -- 172.21.15.69:0/4262417093 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 513136572 0 0) 0x7f99b4001110 con 0x7f99b8054520 2024-04-25T10:59:24.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.598+0000 7f99bf7fe640 1 -- 172.21.15.69:0/4262417093 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99b0003520 con 0x7f99b8054520 2024-04-25T10:59:24.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.598+0000 7f99c538b640 1 -- 172.21.15.69:0/4262417093 >> v2:172.21.15.146:3300/0 conn(0x7f99b8050e00 msgr2=0x7f99b8051210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:24.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.597+0000 7f796effd640 1 --2- 172.21.15.69:0/3673058945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f797006d5d0 0x7f797006d9e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7958009870 tx=0x7f795802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9beb0ea75a745e65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.598+0000 7f796e7fc640 1 -- 172.21.15.69:0/3673058945 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7958037070 con 0x7f797006d5d0 2024-04-25T10:59:24.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.598+0000 7f796e7fc640 1 -- 172.21.15.69:0/3673058945 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7958032440 con 0x7f797006d5d0 2024-04-25T10:59:24.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.598+0000 7f796e7fc640 1 -- 172.21.15.69:0/3673058945 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7958032810 con 0x7f797006d5d0 2024-04-25T10:59:24.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.598+0000 7f99c538b640 1 --2- 172.21.15.69:0/4262417093 >> v2:172.21.15.146:3300/0 conn(0x7f99b8050e00 0x7f99b8051210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.598+0000 7f99c538b640 1 -- 172.21.15.69:0/4262417093 >> v1:172.21.15.146:6789/0 conn(0x7f99b8054520 legacy=0x7f99b80087b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.598+0000 7f99c538b640 1 -- 172.21.15.69:0/4262417093 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b8009100 con 0x7f99b8039ec0 2024-04-25T10:59:24.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.598+0000 7f99bffff640 1 --2- 172.21.15.69:0/4262417093 >> v2:172.21.15.146:3300/0 conn(0x7f99b8050e00 0x7f99b8051210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:59:24.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.598+0000 7f3fb62e0640 1 Processor -- start 2024-04-25T10:59:24.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.598+0000 7f6cef7fe640 1 --2- 172.21.15.69:0/3377156737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ce80dfd90 0x7f6ce80e01a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800be10 tx=0x7f6cdc02a070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d01c5110a5b6bca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.598+0000 7f71822d4640 1 -- 172.21.15.69:0/451654147 >> v2:172.21.15.146:3300/0 conn(0x7f71740e5b10 msgr2=0x7f7174009f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:24.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.598+0000 7f71822d4640 1 --2- 172.21.15.69:0/451654147 >> v2:172.21.15.146:3300/0 conn(0x7f71740e5b10 0x7f7174009f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.598+0000 7f71822d4640 1 -- 172.21.15.69:0/451654147 >> v1:172.21.15.69:6789/0 conn(0x7f717400a4b0 legacy=0x7f71740e4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.598+0000 7f71812d2640 1 -- 172.21.15.69:0/451654147 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2278975328 0 0) 0x7f717c076c60 con 0x7f717400a4b0 2024-04-25T10:59:24.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.599+0000 7f6ceeffd640 1 -- 172.21.15.69:0/3377156737 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc037070 con 0x7f6ce80dfd90 2024-04-25T10:59:24.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.598+0000 7f71822d4640 1 -- 172.21.15.69:0/451654147 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7174052060 con 0x7f7174056440 2024-04-25T10:59:24.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.598+0000 7f7181ad3640 1 --2- 172.21.15.69:0/451654147 >> v2:172.21.15.146:3300/0 conn(0x7f71740e5b10 0x7f7174009f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:59:24.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.599+0000 7f6ceeffd640 1 -- 172.21.15.69:0/3377156737 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cdc02adc0 con 0x7f6ce80dfd90 2024-04-25T10:59:24.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.599+0000 7f6ceeffd640 1 -- 172.21.15.69:0/3377156737 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc032660 con 0x7f6ce80dfd90 2024-04-25T10:59:24.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.599+0000 7f99c538b640 1 --2- 172.21.15.69:0/4262417093 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99b8039ec0 0x7f99b803a2d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f99c00514f0 tx=0x7f99c00695d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13d6080bc1fccd78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.598+0000 7f80bd59a640 1 -- 172.21.15.69:0/2945299897 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2439471445 0 0) 0x7f80c00d1ee0 con 0x7f80c00b75b0 2024-04-25T10:59:24.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.598+0000 7f80bd59a640 1 -- 172.21.15.69:0/2945299897 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80a8003520 con 0x7f80c00b75b0 2024-04-25T10:59:24.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.598+0000 7f80bdd9b640 1 -- 172.21.15.69:0/2945299897 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80c00b46d0 msgr2=0x7f80c01646b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:59:24.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.598+0000 7f80bdd9b640 1 --2- 172.21.15.69:0/2945299897 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80c00b46d0 0x7f80c01646b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.598+0000 7f80bdd9b640 1 -- 172.21.15.69:0/2945299897 >> v1:172.21.15.146:6789/0 conn(0x7f80c00b75b0 legacy=0x7f80c00b79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.598+0000 7f3fb62e0640 1 -- start start 2024-04-25T10:59:24.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.599+0000 7f7975dee640 1 -- 172.21.15.69:0/3673058945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f797006d5d0 msgr2=0x7f797006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.598+0000 7f80bdd9b640 1 -- 172.21.15.69:0/2945299897 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c00d03d0 con 0x7f80c00d4e90 2024-04-25T10:59:24.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.599+0000 7f7975dee640 1 --2- 172.21.15.69:0/3673058945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f797006d5d0 0x7f797006d9e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7958009870 tx=0x7f795802a1c0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.599+0000 7f7975dee640 1 -- 172.21.15.69:0/3673058945 shutdown_connections 2024-04-25T10:59:24.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.599+0000 7f7975dee640 1 --2- 172.21.15.69:0/3673058945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f797006d5d0 0x7f797006d9e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.599+0000 7f7975dee640 1 --2- 172.21.15.69:0/3673058945 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f797014dd70 0x7f7970070f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.599+0000 7f7975dee640 1 -- 172.21.15.69:0/3673058945 >> 172.21.15.69:0/3673058945 conn(0x7f7970073050 msgr2=0x7f7970073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:24.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.599+0000 7faef77fe640 1 -- 172.21.15.69:0/1344350051 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec037070 con 0x7faef0048300 2024-04-25T10:59:24.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.599+0000 7faef77fe640 1 -- 172.21.15.69:0/1344350051 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faeec032440 con 0x7faef0048300 2024-04-25T10:59:24.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.599+0000 7faef77fe640 1 -- 172.21.15.69:0/1344350051 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec0327f0 con 0x7faef0048300 2024-04-25T10:59:24.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.599+0000 7f99bf7fe640 1 -- 172.21.15.69:0/4262417093 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c00ae600 con 0x7f99b8039ec0 2024-04-25T10:59:24.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.599+0000 7f99bf7fe640 1 -- 172.21.15.69:0/4262417093 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99c00ae7a0 con 0x7f99b8039ec0 2024-04-25T10:59:24.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.599+0000 7f99bf7fe640 1 -- 172.21.15.69:0/4262417093 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c00ae600 con 0x7f99b8039ec0 2024-04-25T10:59:24.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.599+0000 7f71822d4640 1 --2- 172.21.15.69:0/451654147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7174056440 0x7f71740e2650 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f71740078c0 tx=0x7f717802a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94daddea86d58984 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.599+0000 7f80bdd9b640 1 --2- 172.21.15.69:0/2945299897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80c00d4e90 0x7f80c00d0bd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f80b40097a0 tx=0x7f80b402a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92eae4562ac0651a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.599+0000 7f7975dee640 1 -- 172.21.15.69:0/3673058945 shutdown_connections 2024-04-25T10:59:24.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.599+0000 7f7975dee640 1 -- 172.21.15.69:0/3673058945 wait complete. 2024-04-25T10:59:24.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.600+0000 7f6cf5320640 1 -- 172.21.15.69:0/3377156737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ce80dfd90 msgr2=0x7f6ce80e01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.600+0000 7f6cf5320640 1 --2- 172.21.15.69:0/3377156737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ce80dfd90 0x7f6ce80e01a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800be10 tx=0x7f6cdc02a070 comp rx=0 tx=0).stop 2024-04-25T10:59:24.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.600+0000 7f71812d2640 1 -- 172.21.15.69:0/451654147 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7178037070 con 0x7f7174056440 2024-04-25T10:59:24.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.600+0000 7f71812d2640 1 -- 172.21.15.69:0/451654147 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f717802adf0 con 0x7f7174056440 2024-04-25T10:59:24.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.600+0000 7f71812d2640 1 -- 172.21.15.69:0/451654147 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7178032660 con 0x7f7174056440 2024-04-25T10:59:24.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.599+0000 7fc46199d640 1 Processor -- start 2024-04-25T10:59:24.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.599+0000 7fc46199d640 1 -- start start 2024-04-25T10:59:24.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.600+0000 7f6cf5320640 1 -- 172.21.15.69:0/3377156737 shutdown_connections 2024-04-25T10:59:24.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.600+0000 7f6cf5320640 1 --2- 172.21.15.69:0/3377156737 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ce80e06e0 0x7f6ce80e3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.600+0000 7f6cf5320640 1 --2- 172.21.15.69:0/3377156737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ce80dfd90 0x7f6ce80e01a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.600+0000 7f6cf5320640 1 --2- 172.21.15.69:0/3377156737 >> v2:172.21.15.69:3300/0 conn(0x7f6ce80340c0 0x7f6ce8006aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.600+0000 7f6cf5320640 1 -- 172.21.15.69:0/3377156737 >> 172.21.15.69:0/3377156737 conn(0x7f6ce8054cf0 msgr2=0x7f6ce8055100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:24.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.600+0000 7f6cf5320640 1 -- 172.21.15.69:0/3377156737 shutdown_connections 2024-04-25T10:59:24.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.600+0000 7f6cf5320640 1 -- 172.21.15.69:0/3377156737 wait complete. 2024-04-25T10:59:24.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.600+0000 7faefdb55640 1 -- 172.21.15.69:0/1344350051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faef0048300 msgr2=0x7faef00065e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.600+0000 7faefdb55640 1 --2- 172.21.15.69:0/1344350051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faef0048300 0x7faef00065e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faeec0097a0 tx=0x7faeec02a1c0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.600+0000 7f99c6dee640 1 -- 172.21.15.69:0/4262417093 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99b8039ec0 msgr2=0x7f99b803a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.600+0000 7f99c6dee640 1 --2- 172.21.15.69:0/4262417093 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99b8039ec0 0x7f99b803a2d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f99c00514f0 tx=0x7f99c00695d0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.601+0000 7f99c6dee640 1 -- 172.21.15.69:0/4262417093 shutdown_connections 2024-04-25T10:59:24.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.601+0000 7f99c6dee640 1 --2- 172.21.15.69:0/4262417093 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99b8039ec0 0x7f99b803a2d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.601+0000 7f99c6dee640 1 --2- 172.21.15.69:0/4262417093 >> v2:172.21.15.146:3300/0 conn(0x7f99b8050e00 0x7f99b8051210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.601+0000 7f99c6dee640 1 -- 172.21.15.69:0/4262417093 >> 172.21.15.69:0/4262417093 conn(0x7f99b8055e30 msgr2=0x7f99b8056240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:24.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.601+0000 7f99c6dee640 1 -- 172.21.15.69:0/4262417093 shutdown_connections 2024-04-25T10:59:24.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.601+0000 7f99c6dee640 1 -- 172.21.15.69:0/4262417093 wait complete. 2024-04-25T10:59:24.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.600+0000 7f3fb62e0640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f3fa8047db0 0x7f3fa80e7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.600+0000 7f3fb62e0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3fa8034060 0x7f3fa80065e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.600+0000 7f3fb62e0640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fa8006130 con 0x7f3fa8048300 2024-04-25T10:59:24.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.600+0000 7f3fb62e0640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f3fa804f690 con 0x7f3fa8047db0 2024-04-25T10:59:24.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.600+0000 7f3fb62e0640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3fa804ffa0 con 0x7f3fa8034060 2024-04-25T10:59:24.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.601+0000 7f3faffff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f3fa8047db0 0x7f3fa80e7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.601+0000 7f3faffff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f3fa8047db0 0x7f3fa80e7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41404/0 (socket says 172.21.15.69:41404) 2024-04-25T10:59:24.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.601+0000 7f3faffff640 1 -- 172.21.15.69:0/3454204860 learned_addr learned my addr 172.21.15.69:0/3454204860 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:24.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.600+0000 7f80bd59a640 1 -- 172.21.15.69:0/2945299897 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b4037070 con 0x7f80c00d4e90 2024-04-25T10:59:24.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.600+0000 7f80bd59a640 1 -- 172.21.15.69:0/2945299897 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80b4032440 con 0x7f80c00d4e90 2024-04-25T10:59:24.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.600+0000 7f80bd59a640 1 -- 172.21.15.69:0/2945299897 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b402ae60 con 0x7f80c00d4e90 2024-04-25T10:59:24.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.601+0000 7f71832d6640 1 -- 172.21.15.69:0/451654147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7174056440 msgr2=0x7f71740e2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.601+0000 7f71832d6640 1 --2- 172.21.15.69:0/451654147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7174056440 0x7f71740e2650 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f71740078c0 tx=0x7f717802a0a0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.601+0000 7f71832d6640 1 -- 172.21.15.69:0/451654147 shutdown_connections 2024-04-25T10:59:24.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.601+0000 7f71832d6640 1 --2- 172.21.15.69:0/451654147 >> v2:172.21.15.146:3300/0 conn(0x7f71740e5b10 0x7f7174009f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.601+0000 7f3faffff640 1 -- 172.21.15.69:0/3454204860 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3fa8034060 msgr2=0x7f3fa80065e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:59:24.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.601+0000 7f3faffff640 1 --2- 172.21.15.69:0/3454204860 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3fa8034060 0x7f3fa80065e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.601+0000 7f3faffff640 1 -- 172.21.15.69:0/3454204860 >> v1:172.21.15.146:6789/0 conn(0x7f3fa8048300 legacy=0x7f3fa8033950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:24.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.601+0000 7f71832d6640 1 --2- 172.21.15.69:0/451654147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7174056440 0x7f71740e2650 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.601+0000 7f71832d6640 1 -- 172.21.15.69:0/451654147 >> 172.21.15.69:0/451654147 conn(0x7f717400c3f0 msgr2=0x7f717400c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:24.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.602+0000 7fc46199d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc454006090 0x7fc454102a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.602+0000 7fc46199d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc45404fb30 0x7fc454102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.602+0000 7fc46199d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc454104f00 0x7fc45404b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.602+0000 7fc45bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc45404fb30 0x7fc454102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.602+0000 7fc46199d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc454050890 con 0x7fc45404fb30 2024-04-25T10:59:24.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.602+0000 7fc45bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc45404fb30 0x7fc454102f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41420/0 (socket says 172.21.15.69:41420) 2024-04-25T10:59:24.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.602+0000 7fc45bfff640 1 -- 172.21.15.69:0/1868292832 learned_addr learned my addr 172.21.15.69:0/1868292832 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:24.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.602+0000 7fc46199d640 1 -- 172.21.15.69:0/1868292832 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc4540541f0 con 0x7fc454104f00 2024-04-25T10:59:24.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.602+0000 7f71832d6640 1 -- 172.21.15.69:0/451654147 shutdown_connections 2024-04-25T10:59:24.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.602+0000 7f71832d6640 1 -- 172.21.15.69:0/451654147 wait complete. 2024-04-25T10:59:24.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.602+0000 7fc46199d640 1 -- 172.21.15.69:0/1868292832 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc454054710 con 0x7fc454006090 2024-04-25T10:59:24.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.602+0000 7f80bf59e640 1 -- 172.21.15.69:0/2945299897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80c00d4e90 msgr2=0x7f80c00d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.602+0000 7f80bf59e640 1 --2- 172.21.15.69:0/2945299897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80c00d4e90 0x7f80c00d0bd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f80b40097a0 tx=0x7f80b402a110 comp rx=0 tx=0).stop 2024-04-25T10:59:24.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.602+0000 7f80bf59e640 1 -- 172.21.15.69:0/2945299897 shutdown_connections 2024-04-25T10:59:24.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.602+0000 7f80bf59e640 1 --2- 172.21.15.69:0/2945299897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80c00d4e90 0x7f80c00d0bd0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.602+0000 7f80bf59e640 1 --2- 172.21.15.69:0/2945299897 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80c00b46d0 0x7f80c01646b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.602+0000 7f80bf59e640 1 -- 172.21.15.69:0/2945299897 >> 172.21.15.69:0/2945299897 conn(0x7f80c00d25b0 msgr2=0x7f80c00d29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:24.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.602+0000 7faefdb55640 1 -- 172.21.15.69:0/1344350051 shutdown_connections 2024-04-25T10:59:24.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.601+0000 7f3faffff640 1 -- 172.21.15.69:0/3454204860 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa8050280 con 0x7f3fa8047db0 2024-04-25T10:59:24.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.602+0000 7f7975dee640 1 Processor -- start 2024-04-25T10:59:24.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.602+0000 7fc46119c640 1 --2- 172.21.15.69:0/1868292832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc454104f00 0x7fc45404b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.602+0000 7f80bf59e640 1 -- 172.21.15.69:0/2945299897 shutdown_connections 2024-04-25T10:59:24.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.602+0000 7f7975dee640 1 -- start start 2024-04-25T10:59:24.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.603+0000 7fc46099b640 1 --2- 172.21.15.69:0/1868292832 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc454006090 0x7fc454102a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.603+0000 7f6cf5320640 1 Processor -- start 2024-04-25T10:59:24.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.603+0000 7fc46119c640 1 -- 172.21.15.69:0/1868292832 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc454006090 msgr2=0x7fc454102a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.603+0000 7fc46119c640 1 --2- 172.21.15.69:0/1868292832 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc454006090 0x7fc454102a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.603+0000 7fc46119c640 1 -- 172.21.15.69:0/1868292832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc45404fb30 msgr2=0x7fc454102f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.603+0000 7fc46119c640 1 --2- 172.21.15.69:0/1868292832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc45404fb30 0x7fc454102f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.603+0000 7fc46119c640 1 -- 172.21.15.69:0/1868292832 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4540f3360 con 0x7fc454104f00 2024-04-25T10:59:24.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.603+0000 7f6cf5320640 1 -- start start 2024-04-25T10:59:24.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.603+0000 7fc46099b640 1 --2- 172.21.15.69:0/1868292832 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc454006090 0x7fc454102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:24.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.604+0000 7f99c6dee640 1 Processor -- start 2024-04-25T10:59:24.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.604+0000 7f99c6dee640 1 -- start start 2024-04-25T10:59:24.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.603+0000 7fc46119c640 1 --2- 172.21.15.69:0/1868292832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc454104f00 0x7fc45404b1d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc45c06bc20 tx=0x7fc45c075dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.603+0000 7fc4597fa640 1 -- 172.21.15.69:0/1868292832 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c0a4040 con 0x7fc454104f00 2024-04-25T10:59:24.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.603+0000 7fc4597fa640 1 -- 172.21.15.69:0/1868292832 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc45c09e6d0 con 0x7fc454104f00 2024-04-25T10:59:24.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.605+0000 7fc4597fa640 1 -- 172.21.15.69:0/1868292832 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c09ea80 con 0x7fc454104f00 2024-04-25T10:59:24.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.605+0000 7fc46199d640 1 -- 172.21.15.69:0/1868292832 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4540f3640 con 0x7fc454104f00 2024-04-25T10:59:24.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.605+0000 7fc46199d640 1 -- 172.21.15.69:0/1868292832 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc45404acb0 con 0x7fc454104f00 2024-04-25T10:59:24.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.605+0000 7f6cf5320640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ce80340c0 0x7f6ce804dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.605+0000 7f6cf5320640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ce80dfd90 0x7f6ce8172c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.605+0000 7f6cf5320640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ce80e06e0 0x7f6ce8173150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.605+0000 7f6cf5320640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce800c450 con 0x7f6ce80340c0 2024-04-25T10:59:24.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.605+0000 7f6cf5320640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce800c2d0 con 0x7f6ce80dfd90 2024-04-25T10:59:24.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.605+0000 7f6cf5320640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6ce80e38c0 con 0x7f6ce80e06e0 2024-04-25T10:59:24.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.605+0000 7f6ceffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ce80340c0 0x7f6ce804dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.605+0000 7f6ceffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ce80340c0 0x7f6ce804dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41430/0 (socket says 172.21.15.69:41430) 2024-04-25T10:59:24.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.605+0000 7f6ceffff640 1 -- 172.21.15.69:0/2259912045 learned_addr learned my addr 172.21.15.69:0/2259912045 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:24.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.605+0000 7f71832d6640 1 Processor -- start 2024-04-25T10:59:24.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.605+0000 7f71832d6640 1 -- start start 2024-04-25T10:59:24.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.604+0000 7f7975dee640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f797006d5d0 0x7f79701593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.604+0000 7f7975dee640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f797014dd70 0x7f7970155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.605+0000 7f7975dee640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7970168250 0x7f797006f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.605+0000 7f796effd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f797014dd70 0x7f7970155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.605+0000 7f796effd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f797014dd70 0x7f7970155ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.69:51916/0 (socket says 172.21.15.69:51916) 2024-04-25T10:59:24.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.605+0000 7f796effd640 1 -- 172.21.15.69:0/2194772549 learned_addr learned my addr 172.21.15.69:0/2194772549 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:24.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.605+0000 7f7975dee640 1 -- 172.21.15.69:0/2194772549 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f79700b5180 con 0x7f797006d5d0 2024-04-25T10:59:24.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.605+0000 7f7975dee640 1 -- 172.21.15.69:0/2194772549 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f79700b4ea0 con 0x7f797014dd70 2024-04-25T10:59:24.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.605+0000 7f7975dee640 1 -- 172.21.15.69:0/2194772549 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f79700b5460 con 0x7f7970168250 2024-04-25T10:59:24.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.605+0000 7f796ffff640 1 --2- 172.21.15.69:0/2194772549 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7970168250 0x7f797006f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.605+0000 7f796effd640 1 -- 172.21.15.69:0/2194772549 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7970168250 msgr2=0x7f797006f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.605+0000 7f796effd640 1 --2- 172.21.15.69:0/2194772549 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7970168250 0x7f797006f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.605+0000 7f796effd640 1 -- 172.21.15.69:0/2194772549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f797006d5d0 msgr2=0x7f79701593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.605+0000 7f796f7fe640 1 --2- 172.21.15.69:0/2194772549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f797006d5d0 0x7f79701593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.605+0000 7f796effd640 1 --2- 172.21.15.69:0/2194772549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f797006d5d0 0x7f79701593a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.605+0000 7f796effd640 1 -- 172.21.15.69:0/2194772549 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7970153c00 con 0x7f797014dd70 2024-04-25T10:59:24.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.605+0000 7f796f7fe640 1 --2- 172.21.15.69:0/2194772549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f797006d5d0 0x7f79701593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:24.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.602+0000 7faefdb55640 1 --2- 172.21.15.69:0/1344350051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faef0048300 0x7faef00065e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.602+0000 7faefdb55640 1 --2- 172.21.15.69:0/1344350051 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faef0034060 0x7faef00e7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.602+0000 7faefdb55640 1 -- 172.21.15.69:0/1344350051 >> 172.21.15.69:0/1344350051 conn(0x7faef0054cf0 msgr2=0x7faef0055100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:24.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.602+0000 7faefdb55640 1 -- 172.21.15.69:0/1344350051 shutdown_connections 2024-04-25T10:59:24.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.602+0000 7faefdb55640 1 -- 172.21.15.69:0/1344350051 wait complete. 2024-04-25T10:59:24.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.602+0000 7f80bf59e640 1 -- 172.21.15.69:0/2945299897 wait complete. 2024-04-25T10:59:24.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.605+0000 7f80bf59e640 1 Processor -- start 2024-04-25T10:59:24.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.605+0000 7f80bf59e640 1 -- start start 2024-04-25T10:59:24.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.603+0000 7f3faffff640 1 --2- 172.21.15.69:0/3454204860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fa8047db0 0x7f3fa80e7d50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3fa800d9b0 tx=0x7f3fa402a450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93736fd5319ab55c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.604+0000 7f3faeffd640 1 -- 172.21.15.69:0/3454204860 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4036070 con 0x7f3fa8047db0 2024-04-25T10:59:24.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.606+0000 7faefdb55640 1 Processor -- start 2024-04-25T10:59:24.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.606+0000 7f796effd640 1 --2- 172.21.15.69:0/2194772549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f797014dd70 0x7f7970155ad0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7958000c00 tx=0x7f79580046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.606+0000 7f6ceffff640 1 -- 172.21.15.69:0/2259912045 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ce80e06e0 msgr2=0x7f6ce8173150 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:59:24.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.606+0000 7faefdb55640 1 -- start start 2024-04-25T10:59:24.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.606+0000 7f6ceffff640 1 --2- 172.21.15.69:0/2259912045 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ce80e06e0 0x7f6ce8173150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.606+0000 7f6ceffff640 1 -- 172.21.15.69:0/2259912045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ce80dfd90 msgr2=0x7f6ce8172c10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:59:24.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.606+0000 7f6ceffff640 1 --2- 172.21.15.69:0/2259912045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ce80dfd90 0x7f6ce8172c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.606+0000 7f6ceffff640 1 -- 172.21.15.69:0/2259912045 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce80ca700 con 0x7f6ce80340c0 2024-04-25T10:59:24.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.606+0000 7fc46199d640 1 -- 172.21.15.69:0/1868292832 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc454005570 con 0x7fc454104f00 2024-04-25T10:59:24.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.604+0000 7f3faeffd640 1 -- 172.21.15.69:0/3454204860 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fa4031600 con 0x7f3fa8047db0 2024-04-25T10:59:24.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.605+0000 7f3fb62e0640 1 -- 172.21.15.69:0/3454204860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fa8047db0 msgr2=0x7f3fa80e7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.605+0000 7f3fb62e0640 1 --2- 172.21.15.69:0/3454204860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fa8047db0 0x7f3fa80e7d50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3fa800d9b0 tx=0x7f3fa402a450 comp rx=0 tx=0).stop 2024-04-25T10:59:24.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.607+0000 7f6ceffff640 1 --2- 172.21.15.69:0/2259912045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ce80340c0 0x7f6ce804dad0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400c820 tx=0x7f6ce400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.606+0000 7f7967fff640 1 -- 172.21.15.69:0/2194772549 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7958037040 con 0x7f797014dd70 2024-04-25T10:59:24.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.606+0000 7f7967fff640 1 -- 172.21.15.69:0/2194772549 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7958042070 con 0x7f797014dd70 2024-04-25T10:59:24.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.606+0000 7f7967fff640 1 -- 172.21.15.69:0/2194772549 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7958030940 con 0x7f797014dd70 2024-04-25T10:59:24.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.606+0000 7f7975dee640 1 -- 172.21.15.69:0/2194772549 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7970156560 con 0x7f797014dd70 2024-04-25T10:59:24.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.606+0000 7f7975dee640 1 -- 172.21.15.69:0/2194772549 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7970153ec0 con 0x7f797014dd70 2024-04-25T10:59:24.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.607+0000 7f99c6dee640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99b8039ec0 0x7f99b80d7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.607+0000 7f99c6dee640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99b8050e00 0x7f99b8178b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.607+0000 7f99c6dee640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99b8054520 0x7f99b8179040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.607+0000 7f6cecff9640 1 -- 172.21.15.69:0/2259912045 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4015070 con 0x7f6ce80340c0 2024-04-25T10:59:24.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.607+0000 7f71832d6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f717400a4b0 0x7f7174007d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.607+0000 7f99c4b8a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99b8039ec0 0x7f99b80d7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.607+0000 7f99c4b8a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99b8039ec0 0x7f99b80d7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.69:58334/0 (socket says 172.21.15.69:58334) 2024-04-25T10:59:24.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.607+0000 7f99bffff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99b8050e00 0x7f99b8178b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.607+0000 7f6cecff9640 1 -- 172.21.15.69:0/2259912045 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ce4007830 con 0x7f6ce80340c0 2024-04-25T10:59:24.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.607+0000 7f6cecff9640 1 -- 172.21.15.69:0/2259912045 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4007c20 con 0x7f6ce80340c0 2024-04-25T10:59:24.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.607+0000 7f6cf5320640 1 -- 172.21.15.69:0/2259912045 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ce80ca9e0 con 0x7f6ce80340c0 2024-04-25T10:59:24.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.607+0000 7f99bffff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99b8050e00 0x7f99b8178b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.69:51926/0 (socket says 172.21.15.69:51926) 2024-04-25T10:59:24.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.607+0000 7f6cf5320640 1 -- 172.21.15.69:0/2259912045 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ce80cd9a0 con 0x7f6ce80340c0 2024-04-25T10:59:24.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.608+0000 7f99c4b8a640 1 -- 172.21.15.69:0/3450251409 learned_addr learned my addr 172.21.15.69:0/3450251409 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:24.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.608+0000 7f99c538b640 1 --2- 172.21.15.69:0/3450251409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99b8054520 0x7f99b8179040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.608+0000 7f99c6dee640 1 -- 172.21.15.69:0/3450251409 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f99b80096c0 con 0x7f99b8054520 2024-04-25T10:59:24.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.608+0000 7f99c6dee640 1 -- 172.21.15.69:0/3450251409 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f99b80396c0 con 0x7f99b8050e00 2024-04-25T10:59:24.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.608+0000 7f3fb62e0640 1 -- 172.21.15.69:0/3454204860 shutdown_connections 2024-04-25T10:59:24.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.608+0000 7f3fb62e0640 1 --2- 172.21.15.69:0/3454204860 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3fa8034060 0x7f3fa80065e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.608+0000 7f3fb62e0640 1 --2- 172.21.15.69:0/3454204860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fa8047db0 0x7f3fa80e7d50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.608+0000 7f3fb62e0640 1 -- 172.21.15.69:0/3454204860 >> 172.21.15.69:0/3454204860 conn(0x7f3fa8054cf0 msgr2=0x7f3fa8055100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:24.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.608+0000 7faefdb55640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faef0047db0 0x7faef00e4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.608+0000 7f80bf59e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80c00b46d0 0x7f80c01604a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.608+0000 7f80bf59e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80c00b75b0 0x7f80c01fe2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.608+0000 7f80bf59e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80c00d4e90 0x7f80c01fe7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.608+0000 7f80bf59e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f80c00ba1c0 con 0x7f80c00b46d0 2024-04-25T10:59:24.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.608+0000 7f7965ffb640 1 -- 172.21.15.69:0/2194772549 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7970050740 con 0x7f797014dd70 2024-04-25T10:59:24.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.608+0000 7f80bf59e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f80c00b8c50 con 0x7f80c00b75b0 2024-04-25T10:59:24.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.608+0000 7f80bf59e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f80c00ba340 con 0x7f80c00d4e90 2024-04-25T10:59:24.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.608+0000 7faefdb55640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faef0048300 0x7faef004cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.608+0000 7faefdb55640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faef0053b40 0x7faef00e20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.608+0000 7faefdb55640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7faef00509c0 con 0x7faef0053b40 2024-04-25T10:59:24.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.608+0000 7faefdb55640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7faef0050840 con 0x7faef0047db0 2024-04-25T10:59:24.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.608+0000 7faefdb55640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7faef0006b20 con 0x7faef0048300 2024-04-25T10:59:24.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.608+0000 7faefd354640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faef0053b40 0x7faef00e20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.608+0000 7f80be59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80c00b46d0 0x7f80c01604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.608+0000 7f80be59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80c00b46d0 0x7f80c01604a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41448/0 (socket says 172.21.15.69:41448) 2024-04-25T10:59:24.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.608+0000 7f3fb62e0640 1 -- 172.21.15.69:0/3454204860 shutdown_connections 2024-04-25T10:59:24.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.609+0000 7f80be59c640 1 -- 172.21.15.69:0/691542650 learned_addr learned my addr 172.21.15.69:0/691542650 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:24.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.609+0000 7faefcb53640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faef0047db0 0x7faef00e4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.609+0000 7fc4597fa640 1 -- 172.21.15.69:0/1868292832 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc45c0b1030 con 0x7fc454104f00 2024-04-25T10:59:24.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.609+0000 7f80bed9d640 1 --2- 172.21.15.69:0/691542650 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80c00d4e90 0x7f80c01fe7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.609+0000 7faef7fff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faef0048300 0x7faef004cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.609+0000 7faef7fff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faef0048300 0x7faef004cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.69:58348/0 (socket says 172.21.15.69:58348) 2024-04-25T10:59:24.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.609+0000 7faef7fff640 1 -- 172.21.15.69:0/688089409 learned_addr learned my addr 172.21.15.69:0/688089409 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:24.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.609+0000 7faefcb53640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faef0047db0 0x7faef00e4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.69:51950/0 (socket says 172.21.15.69:51950) 2024-04-25T10:59:24.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.608+0000 7f3fb62e0640 1 -- 172.21.15.69:0/3454204860 wait complete. 2024-04-25T10:59:24.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.609+0000 7f80bdd9b640 1 --2- 172.21.15.69:0/691542650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80c00b75b0 0x7f80c01fe2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.609+0000 7f80be59c640 1 -- 172.21.15.69:0/691542650 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80c00d4e90 msgr2=0x7f80c01fe7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.609+0000 7f80be59c640 1 --2- 172.21.15.69:0/691542650 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80c00d4e90 0x7f80c01fe7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.609+0000 7f80be59c640 1 -- 172.21.15.69:0/691542650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80c00b75b0 msgr2=0x7f80c01fe2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.609+0000 7f80be59c640 1 --2- 172.21.15.69:0/691542650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80c00b75b0 0x7f80c01fe2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.609+0000 7f80be59c640 1 -- 172.21.15.69:0/691542650 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c0073f20 con 0x7f80c00b46d0 2024-04-25T10:59:24.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.609+0000 7f6cf5320640 1 -- 172.21.15.69:0/2259912045 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ce8005570 con 0x7f6ce80340c0 2024-04-25T10:59:24.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.609+0000 7fc4597fa640 1 --2- 172.21.15.69:0/1868292832 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc440074530 0x7fc440076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.609+0000 7f80be59c640 1 --2- 172.21.15.69:0/691542650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80c00b46d0 0x7f80c01604a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f80b000b410 tx=0x7f80b000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.610+0000 7fc46099b640 1 --2- 172.21.15.69:0/1868292832 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc440074530 0x7fc440076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.610+0000 7fc4597fa640 1 -- 172.21.15.69:0/1868292832 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc45c1214a0 con 0x7fc454104f00 2024-04-25T10:59:24.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.610+0000 7faefd354640 1 -- 172.21.15.69:0/688089409 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faef0048300 msgr2=0x7faef004cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.610+0000 7faefd354640 1 --2- 172.21.15.69:0/688089409 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faef0048300 0x7faef004cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.610+0000 7faefd354640 1 -- 172.21.15.69:0/688089409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faef0047db0 msgr2=0x7faef00e4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.610+0000 7faefd354640 1 --2- 172.21.15.69:0/688089409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faef0047db0 0x7faef00e4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.610+0000 7fc46099b640 1 --2- 172.21.15.69:0/1868292832 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc440074530 0x7fc440076a40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc44800a810 tx=0x7fc448008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.611+0000 7f6cecff9640 1 -- 172.21.15.69:0/2259912045 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6ce4018070 con 0x7f6ce80340c0 2024-04-25T10:59:24.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.610+0000 7faefd354640 1 -- 172.21.15.69:0/688089409 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef0054040 con 0x7faef0053b40 2024-04-25T10:59:24.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.610+0000 7faefd354640 1 --2- 172.21.15.69:0/688089409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faef0053b40 0x7faef00e20d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7faeec031000 tx=0x7faeec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.610+0000 7faef57fa640 1 -- 172.21.15.69:0/688089409 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec037050 con 0x7faef0053b40 2024-04-25T10:59:24.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.610+0000 7faefdb55640 1 -- 172.21.15.69:0/688089409 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faef00487b0 con 0x7faef0053b40 2024-04-25T10:59:24.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.610+0000 7faef57fa640 1 -- 172.21.15.69:0/688089409 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faeec043070 con 0x7faef0053b40 2024-04-25T10:59:24.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.610+0000 7faef57fa640 1 -- 172.21.15.69:0/688089409 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec0325f0 con 0x7faef0053b40 2024-04-25T10:59:24.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.611+0000 7faefdb55640 1 -- 172.21.15.69:0/688089409 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faef0048d30 con 0x7faef0053b40 2024-04-25T10:59:24.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.610+0000 7f80aeffd640 1 -- 172.21.15.69:0/691542650 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b0015020 con 0x7f80c00b46d0 2024-04-25T10:59:24.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.610+0000 7f80aeffd640 1 -- 172.21.15.69:0/691542650 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80b0011830 con 0x7f80c00b46d0 2024-04-25T10:59:24.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.610+0000 7f80aeffd640 1 -- 172.21.15.69:0/691542650 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b0011c20 con 0x7f80c00b46d0 2024-04-25T10:59:24.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.610+0000 7f80bf59e640 1 -- 172.21.15.69:0/691542650 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80c01611d0 con 0x7f80c00b46d0 2024-04-25T10:59:24.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.610+0000 7f80bf59e640 1 -- 172.21.15.69:0/691542650 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80c014bbe0 con 0x7f80c00b46d0 2024-04-25T10:59:24.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.611+0000 7f3fb62e0640 1 Processor -- start 2024-04-25T10:59:24.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.612+0000 7f3fb62e0640 1 -- start start 2024-04-25T10:59:24.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.612+0000 7f6cecff9640 1 --2- 172.21.15.69:0/2259912045 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f6cd40744b0 0x7f6cd40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.608+0000 7f71832d6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7174056440 0x7f71740cb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.612+0000 7f6cecff9640 1 -- 172.21.15.69:0/2259912045 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6ce4095240 con 0x7f6ce80340c0 2024-04-25T10:59:24.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.612+0000 7faefdb55640 1 -- 172.21.15.69:0/688089409 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faef0005570 con 0x7faef0053b40 2024-04-25T10:59:24.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.612+0000 7f80bf59e640 1 -- 172.21.15.69:0/691542650 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80a4005290 con 0x7f80c00b46d0 2024-04-25T10:59:24.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.609+0000 7f99c6dee640 1 -- 172.21.15.69:0/3450251409 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f99b80353e0 con 0x7f99b8039ec0 2024-04-25T10:59:24.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.609+0000 7f99bffff640 1 -- 172.21.15.69:0/3450251409 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99b8039ec0 msgr2=0x7f99b80d7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.609+0000 7f99bffff640 1 --2- 172.21.15.69:0/3450251409 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99b8039ec0 0x7f99b80d7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.609+0000 7f99bffff640 1 -- 172.21.15.69:0/3450251409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99b8054520 msgr2=0x7f99b8179040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.609+0000 7f99bffff640 1 --2- 172.21.15.69:0/3450251409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99b8054520 0x7f99b8179040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.610+0000 7f99bffff640 1 -- 172.21.15.69:0/3450251409 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b804a010 con 0x7f99b8050e00 2024-04-25T10:59:24.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.610+0000 7f99c4b8a640 1 --2- 172.21.15.69:0/3450251409 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99b8039ec0 0x7f99b80d7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:24.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.610+0000 7f99bffff640 1 --2- 172.21.15.69:0/3450251409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99b8050e00 0x7f99b8178b00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f99ac012be0 tx=0x7f99ac010490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.612+0000 7f99c538b640 1 --2- 172.21.15.69:0/3450251409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99b8054520 0x7f99b8179040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:24.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.612+0000 7f6cef7fe640 1 --2- 172.21.15.69:0/2259912045 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f6cd40744b0 0x7f6cd40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.610+0000 7f7967fff640 1 -- 172.21.15.69:0/2194772549 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f795802ad10 con 0x7f797014dd70 2024-04-25T10:59:24.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.608+0000 7f71832d6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71740e5b10 0x7f717404d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.608+0000 7f71832d6640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7174052900 con 0x7f717400a4b0 2024-04-25T10:59:24.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.608+0000 7f71832d6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7174052620 con 0x7f71740e5b10 2024-04-25T10:59:24.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.608+0000 7f71832d6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f71740e3d30 con 0x7f7174056440 2024-04-25T10:59:24.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.609+0000 7f7181ad3640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7174056440 0x7f71740cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.609+0000 7f7181ad3640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7174056440 0x7f71740cb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.69:58376/0 (socket says 172.21.15.69:58376) 2024-04-25T10:59:24.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.609+0000 7f7181ad3640 1 -- 172.21.15.69:0/3096217692 learned_addr learned my addr 172.21.15.69:0/3096217692 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:24.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.610+0000 7f7181ad3640 1 -- 172.21.15.69:0/3096217692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71740e5b10 msgr2=0x7f717404d020 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:59:24.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.610+0000 7f71822d4640 1 --2- 172.21.15.69:0/3096217692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f717400a4b0 0x7f7174007d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.612+0000 7f7181ad3640 1 --2- 172.21.15.69:0/3096217692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71740e5b10 0x7f717404d020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.612+0000 7f7181ad3640 1 -- 172.21.15.69:0/3096217692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f717400a4b0 msgr2=0x7f7174007d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:24.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.612+0000 7f7181ad3640 1 --2- 172.21.15.69:0/3096217692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f717400a4b0 0x7f7174007d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.612+0000 7f7181ad3640 1 -- 172.21.15.69:0/3096217692 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7174049720 con 0x7f7174056440 2024-04-25T10:59:24.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.612+0000 7f71822d4640 1 --2- 172.21.15.69:0/3096217692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f717400a4b0 0x7f7174007d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:24.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.610+0000 7f7967fff640 1 --2- 172.21.15.69:0/2194772549 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f79500744b0 0x7f79500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.610+0000 7f7967fff640 1 -- 172.21.15.69:0/2194772549 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f79580b56e0 con 0x7f797014dd70 2024-04-25T10:59:24.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.612+0000 7f796f7fe640 1 --2- 172.21.15.69:0/2194772549 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f79500744b0 0x7f79500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.612+0000 7f7181ad3640 1 --2- 172.21.15.69:0/3096217692 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7174056440 0x7f71740cb090 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f716c00b420 tx=0x7f716c00b8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.613+0000 7f6cef7fe640 1 --2- 172.21.15.69:0/2259912045 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f6cd40744b0 0x7f6cd40769c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800be10 tx=0x7f6cdc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.613+0000 7f7172ffd640 1 -- 172.21.15.69:0/3096217692 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c022070 con 0x7f7174056440 2024-04-25T10:59:24.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.613+0000 7f7172ffd640 1 -- 172.21.15.69:0/3096217692 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f716c01d410 con 0x7f7174056440 2024-04-25T10:59:24.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.613+0000 7f7172ffd640 1 -- 172.21.15.69:0/3096217692 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c01d7c0 con 0x7f7174056440 2024-04-25T10:59:24.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.613+0000 7f71832d6640 1 -- 172.21.15.69:0/3096217692 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7174009910 con 0x7f7174056440 2024-04-25T10:59:24.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.613+0000 7f796f7fe640 1 --2- 172.21.15.69:0/2194772549 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f79500744b0 0x7f79500769c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7960009a20 tx=0x7f7960008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.613+0000 7f71832d6640 1 -- 172.21.15.69:0/3096217692 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f717404cb00 con 0x7f7174056440 2024-04-25T10:59:24.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.613+0000 7fc4597fa640 1 -- 172.21.15.69:0/1868292832 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc45c0b8020 con 0x7fc454104f00 2024-04-25T10:59:24.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.613+0000 7faef57fa640 1 -- 172.21.15.69:0/688089409 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7faeec032790 con 0x7faef0053b40 2024-04-25T10:59:24.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.613+0000 7f3fb62e0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fa8047db0 0x7f3fa80df950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.613+0000 7f3fb62e0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3fa8048300 0x7f3fa804d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.613+0000 7f3fb62e0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fa804c770 0x7f3fa804de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.614+0000 7f3fb62e0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa80509c0 con 0x7f3fa8047db0 2024-04-25T10:59:24.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.614+0000 7f3faffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fa8047db0 0x7f3fa80df950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.614+0000 7f3faffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fa8047db0 0x7f3fa80df950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41472/0 (socket says 172.21.15.69:41472) 2024-04-25T10:59:24.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.614+0000 7f3faffff640 1 -- 172.21.15.69:0/1935169836 learned_addr learned my addr 172.21.15.69:0/1935169836 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:24.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.614+0000 7f3fb62e0640 1 -- 172.21.15.69:0/1935169836 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa8050840 con 0x7f3fa804c770 2024-04-25T10:59:24.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.614+0000 7f3fb62e0640 1 -- 172.21.15.69:0/1935169836 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3fa8006b20 con 0x7f3fa8048300 2024-04-25T10:59:24.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.614+0000 7f71832d6640 1 -- 172.21.15.69:0/3096217692 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7174005570 con 0x7f7174056440 2024-04-25T10:59:24.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.614+0000 7f3faffff640 1 -- 172.21.15.69:0/1935169836 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3fa8048300 msgr2=0x7f3fa804d900 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:59:24.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.614+0000 7f3faffff640 1 --2- 172.21.15.69:0/1935169836 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3fa8048300 0x7f3fa804d900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.614+0000 7f3faffff640 1 -- 172.21.15.69:0/1935169836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fa804c770 msgr2=0x7f3fa804de40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:59:24.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.614+0000 7f3faffff640 1 --2- 172.21.15.69:0/1935169836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fa804c770 0x7f3fa804de40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:24.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.614+0000 7f3faffff640 1 -- 172.21.15.69:0/1935169836 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa8048710 con 0x7f3fa8047db0 2024-04-25T10:59:24.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.614+0000 7faef57fa640 1 --2- 172.21.15.69:0/688089409 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7faedc0744b0 0x7faedc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.614+0000 7faefcb53640 1 --2- 172.21.15.69:0/688089409 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7faedc0744b0 0x7faedc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.615+0000 7f3faffff640 1 --2- 172.21.15.69:0/1935169836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fa8047db0 0x7f3fa80df950 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3fa402aa40 tx=0x7f3fa40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.615+0000 7faefcb53640 1 --2- 172.21.15.69:0/688089409 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7faedc0744b0 0x7faedc0769c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7faef806bc20 tx=0x7faef806a5c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.615+0000 7faef57fa640 1 -- 172.21.15.69:0/688089409 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7faeec0b5a30 con 0x7faef0053b40 2024-04-25T10:59:24.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.615+0000 7f7172ffd640 1 -- 172.21.15.69:0/3096217692 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f716c025070 con 0x7f7174056440 2024-04-25T10:59:24.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.616+0000 7f7172ffd640 1 --2- 172.21.15.69:0/3096217692 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f7160074530 0x7f7160076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.616+0000 7f71822d4640 1 --2- 172.21.15.69:0/3096217692 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f7160074530 0x7f7160076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.616+0000 7f7172ffd640 1 -- 172.21.15.69:0/3096217692 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f716c09cc60 con 0x7f7174056440 2024-04-25T10:59:24.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.616+0000 7f71822d4640 1 --2- 172.21.15.69:0/3096217692 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f7160074530 0x7f7160076a40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7178034000 tx=0x7f717802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.618+0000 7f99bd7fa640 1 -- 172.21.15.69:0/3450251409 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ac01f020 con 0x7f99b8050e00 2024-04-25T10:59:24.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.618+0000 7f99bd7fa640 1 -- 172.21.15.69:0/3450251409 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99ac00d280 con 0x7f99b8050e00 2024-04-25T10:59:24.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.618+0000 7f99c6dee640 1 -- 172.21.15.69:0/3450251409 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99b80d8350 con 0x7f99b8050e00 2024-04-25T10:59:24.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.618+0000 7f99bd7fa640 1 -- 172.21.15.69:0/3450251409 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ac010dc0 con 0x7f99b8050e00 2024-04-25T10:59:24.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.618+0000 7f7967fff640 1 -- 172.21.15.69:0/2194772549 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f795807fe60 con 0x7f797014dd70 2024-04-25T10:59:24.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.619+0000 7f6cecff9640 1 -- 172.21.15.69:0/2259912045 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6ce405f930 con 0x7f6ce80340c0 2024-04-25T10:59:24.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.619+0000 7f99c6dee640 1 -- 172.21.15.69:0/3450251409 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99b804a820 con 0x7f99b8050e00 2024-04-25T10:59:24.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.620+0000 7f99c6dee640 1 -- 172.21.15.69:0/3450251409 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99b80059f0 con 0x7f99b8050e00 2024-04-25T10:59:24.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.621+0000 7f7172ffd640 1 -- 172.21.15.69:0/3096217692 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f716c066f10 con 0x7f7174056440 2024-04-25T10:59:24.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.622+0000 7f99bd7fa640 1 -- 172.21.15.69:0/3450251409 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f99ac022020 con 0x7f99b8050e00 2024-04-25T10:59:24.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.621+0000 7f80aeffd640 1 -- 172.21.15.69:0/691542650 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f80b0018020 con 0x7f80c00b46d0 2024-04-25T10:59:24.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.622+0000 7f80aeffd640 1 --2- 172.21.15.69:0/691542650 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f80a0074500 0x7f80a0076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.622+0000 7f80bdd9b640 1 --2- 172.21.15.69:0/691542650 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f80a0074500 0x7f80a0076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.623+0000 7f99bd7fa640 1 --2- 172.21.15.69:0/3450251409 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f99a40744b0 0x7f99a40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.623+0000 7f80bdd9b640 1 --2- 172.21.15.69:0/691542650 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f80a0074500 0x7f80a0076a10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f80b4034000 tx=0x7f80b402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.623+0000 7f99c4b8a640 1 --2- 172.21.15.69:0/3450251409 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f99a40744b0 0x7f99a40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.624+0000 7f99bd7fa640 1 -- 172.21.15.69:0/3450251409 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f99ac09d1b0 con 0x7f99b8050e00 2024-04-25T10:59:24.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.624+0000 7f99c4b8a640 1 --2- 172.21.15.69:0/3450251409 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f99a40744b0 0x7f99a40769c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f99b4007a20 tx=0x7f99b4007bf0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.625+0000 7f80aeffd640 1 -- 172.21.15.69:0/691542650 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f80b0011dc0 con 0x7f80c00b46d0 2024-04-25T10:59:24.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.625+0000 7f3facff9640 1 -- 172.21.15.69:0/1935169836 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4041020 con 0x7f3fa8047db0 2024-04-25T10:59:24.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.625+0000 7faef57fa640 1 -- 172.21.15.69:0/688089409 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faef00487b0 con 0x7faef0053b40 2024-04-25T10:59:24.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.625+0000 7f3facff9640 1 -- 172.21.15.69:0/1935169836 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fa4003810 con 0x7f3fa8047db0 2024-04-25T10:59:24.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.627+0000 7f3facff9640 1 -- 172.21.15.69:0/1935169836 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4036040 con 0x7f3fa8047db0 2024-04-25T10:59:24.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.627+0000 7f3fb62e0640 1 -- 172.21.15.69:0/1935169836 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fa804b120 con 0x7f3fa8047db0 2024-04-25T10:59:24.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.627+0000 7f3fb62e0640 1 -- 172.21.15.69:0/1935169836 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fa8048920 con 0x7f3fa8047db0 2024-04-25T10:59:24.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.627+0000 7f99bd7fa640 1 -- 172.21.15.69:0/3450251409 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f99ac029030 con 0x7f99b8050e00 2024-04-25T10:59:24.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.628+0000 7f3fb62e0640 1 -- 172.21.15.69:0/1935169836 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fa804b640 con 0x7f3fa8047db0 2024-04-25T10:59:24.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.635+0000 7f80aeffd640 1 -- 172.21.15.69:0/691542650 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f80b001f020 con 0x7f80c00b46d0 2024-04-25T10:59:24.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.638+0000 7f3facff9640 1 -- 172.21.15.69:0/1935169836 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3fa4044020 con 0x7f3fa8047db0 2024-04-25T10:59:24.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.638+0000 7f3facff9640 1 --2- 172.21.15.69:0/1935169836 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3f94074580 0x7f3f94076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:24.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.639+0000 7f3facff9640 1 -- 172.21.15.69:0/1935169836 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3fa40b67c0 con 0x7f3fa8047db0 2024-04-25T10:59:24.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.639+0000 7f3faf7fe640 1 --2- 172.21.15.69:0/1935169836 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3f94074580 0x7f3f94076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:24.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.640+0000 7f3faf7fe640 1 --2- 172.21.15.69:0/1935169836 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3f94074580 0x7f3f94076a90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3fa80e1660 tx=0x7f3f9c006b50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:24.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:24.646+0000 7f3facff9640 1 -- 172.21.15.69:0/1935169836 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3fa804b640 con 0x7f3fa8047db0 2024-04-25T10:59:25.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.024+0000 7fc46199d640 1 -- 172.21.15.69:0/1868292832 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fc4540f3b60 con 0x7fc454104f00 2024-04-25T10:59:25.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.025+0000 7fc4597fa640 1 -- 172.21.15.69:0/1868292832 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc45c098e20 con 0x7fc454104f00 2024-04-25T10:59:25.026 INFO:teuthology.orchestra.run.smithi069.stdout:55834574856 2024-04-25T10:59:25.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.028+0000 7fc446ffd640 1 -- 172.21.15.69:0/1868292832 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc440074530 msgr2=0x7fc440076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.028+0000 7fc446ffd640 1 --2- 172.21.15.69:0/1868292832 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc440074530 0x7fc440076a40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc44800a810 tx=0x7fc448008040 comp rx=0 tx=0).stop 2024-04-25T10:59:25.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.028+0000 7fc446ffd640 1 -- 172.21.15.69:0/1868292832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc454104f00 msgr2=0x7fc45404b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.028+0000 7fc446ffd640 1 --2- 172.21.15.69:0/1868292832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc454104f00 0x7fc45404b1d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc45c06bc20 tx=0x7fc45c075dc0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.029+0000 7fc446ffd640 1 -- 172.21.15.69:0/1868292832 shutdown_connections 2024-04-25T10:59:25.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.029+0000 7fc446ffd640 1 --2- 172.21.15.69:0/1868292832 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc440074530 0x7fc440076a40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.029+0000 7fc446ffd640 1 --2- 172.21.15.69:0/1868292832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc454104f00 0x7fc45404b1d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.029+0000 7fc446ffd640 1 --2- 172.21.15.69:0/1868292832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc45404fb30 0x7fc454102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.029+0000 7fc446ffd640 1 --2- 172.21.15.69:0/1868292832 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc454006090 0x7fc454102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.029+0000 7fc446ffd640 1 -- 172.21.15.69:0/1868292832 >> 172.21.15.69:0/1868292832 conn(0x7fc454054cf0 msgr2=0x7fc4540ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:25.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.029+0000 7fc446ffd640 1 -- 172.21.15.69:0/1868292832 shutdown_connections 2024-04-25T10:59:25.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.029+0000 7fc446ffd640 1 -- 172.21.15.69:0/1868292832 wait complete. 2024-04-25T10:59:25.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.034+0000 7f6cf5320640 1 -- 172.21.15.69:0/2259912045 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f6ce804ba00 con 0x7f6ce80340c0 2024-04-25T10:59:25.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.034+0000 7f6cecff9640 1 -- 172.21.15.69:0/2259912045 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6ce405f230 con 0x7f6ce80340c0 2024-04-25T10:59:25.035 INFO:teuthology.orchestra.run.smithi069.stdout:55834574856 2024-04-25T10:59:25.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.038+0000 7f6cda7fc640 1 -- 172.21.15.69:0/2259912045 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f6cd40744b0 msgr2=0x7f6cd40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.038+0000 7f6cda7fc640 1 --2- 172.21.15.69:0/2259912045 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f6cd40744b0 0x7f6cd40769c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800be10 tx=0x7f6cdc02e000 comp rx=0 tx=0).stop 2024-04-25T10:59:25.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.038+0000 7f6cda7fc640 1 -- 172.21.15.69:0/2259912045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ce80340c0 msgr2=0x7f6ce804dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.038+0000 7f6cda7fc640 1 --2- 172.21.15.69:0/2259912045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ce80340c0 0x7f6ce804dad0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400c820 tx=0x7f6ce400cce0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.038+0000 7f6cda7fc640 1 -- 172.21.15.69:0/2259912045 shutdown_connections 2024-04-25T10:59:25.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.038+0000 7f6cda7fc640 1 --2- 172.21.15.69:0/2259912045 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f6cd40744b0 0x7f6cd40769c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.038+0000 7f6cda7fc640 1 --2- 172.21.15.69:0/2259912045 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ce80e06e0 0x7f6ce8173150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.038+0000 7f6cda7fc640 1 --2- 172.21.15.69:0/2259912045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ce80dfd90 0x7f6ce8172c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.038+0000 7f6cda7fc640 1 --2- 172.21.15.69:0/2259912045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ce80340c0 0x7f6ce804dad0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.038+0000 7f6cda7fc640 1 -- 172.21.15.69:0/2259912045 >> 172.21.15.69:0/2259912045 conn(0x7f6ce8054cf0 msgr2=0x7f6ce804ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:25.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.039+0000 7f6cda7fc640 1 -- 172.21.15.69:0/2259912045 shutdown_connections 2024-04-25T10:59:25.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.039+0000 7f6cda7fc640 1 -- 172.21.15.69:0/2259912045 wait complete. 2024-04-25T10:59:25.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.042+0000 7f7965ffb640 1 -- 172.21.15.69:0/2194772549 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f79700b6500 con 0x7f797014dd70 2024-04-25T10:59:25.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.043+0000 7f7967fff640 1 -- 172.21.15.69:0/2194772549 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f795807f760 con 0x7f797014dd70 2024-04-25T10:59:25.044 INFO:teuthology.orchestra.run.smithi069.stdout:55834574856 2024-04-25T10:59:25.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.046+0000 7f71832d6640 1 -- 172.21.15.69:0/3096217692 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f7174049480 con 0x7f7174056440 2024-04-25T10:59:25.048 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2024-04-25T10:59:25.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.047+0000 7f7172ffd640 1 -- 172.21.15.69:0/3096217692 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f716c06b210 con 0x7f7174056440 2024-04-25T10:59:25.048 DEBUG:teuthology.parallel:result is None 2024-04-25T10:59:25.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.047+0000 7f7975dee640 1 -- 172.21.15.69:0/2194772549 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f79500744b0 msgr2=0x7f79500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.047+0000 7f7975dee640 1 --2- 172.21.15.69:0/2194772549 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f79500744b0 0x7f79500769c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7960009a20 tx=0x7f7960008040 comp rx=0 tx=0).stop 2024-04-25T10:59:25.049 INFO:teuthology.orchestra.run.smithi069.stdout:55834574855 2024-04-25T10:59:25.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.047+0000 7f7975dee640 1 -- 172.21.15.69:0/2194772549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f797014dd70 msgr2=0x7f7970155ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.047+0000 7f7975dee640 1 --2- 172.21.15.69:0/2194772549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f797014dd70 0x7f7970155ad0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7958000c00 tx=0x7f79580046c0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.047+0000 7f7975dee640 1 -- 172.21.15.69:0/2194772549 shutdown_connections 2024-04-25T10:59:25.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.047+0000 7f7975dee640 1 --2- 172.21.15.69:0/2194772549 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f79500744b0 0x7f79500769c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.047+0000 7f7975dee640 1 --2- 172.21.15.69:0/2194772549 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7970168250 0x7f797006f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.047+0000 7f7975dee640 1 --2- 172.21.15.69:0/2194772549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f797014dd70 0x7f7970155ad0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.047+0000 7f7975dee640 1 --2- 172.21.15.69:0/2194772549 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f797006d5d0 0x7f79701593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.047+0000 7f7975dee640 1 -- 172.21.15.69:0/2194772549 >> 172.21.15.69:0/2194772549 conn(0x7f7970073050 msgr2=0x7f79700b16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:25.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.048+0000 7f7975dee640 1 -- 172.21.15.69:0/2194772549 shutdown_connections 2024-04-25T10:59:25.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.048+0000 7f7975dee640 1 -- 172.21.15.69:0/2194772549 wait complete. 2024-04-25T10:59:25.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.048+0000 7f99c6dee640 1 -- 172.21.15.69:0/3450251409 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f99b804a2d0 con 0x7f99b8050e00 2024-04-25T10:59:25.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.049+0000 7f99bd7fa640 1 -- 172.21.15.69:0/3450251409 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f99ac00b070 con 0x7f99b8050e00 2024-04-25T10:59:25.051 INFO:teuthology.orchestra.run.smithi069.stdout:55834574855 2024-04-25T10:59:25.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.050+0000 7f71832d6640 1 -- 172.21.15.69:0/3096217692 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f7160074530 msgr2=0x7f7160076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.050+0000 7f71832d6640 1 --2- 172.21.15.69:0/3096217692 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f7160074530 0x7f7160076a40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7178034000 tx=0x7f717802e000 comp rx=0 tx=0).stop 2024-04-25T10:59:25.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.050+0000 7f71832d6640 1 -- 172.21.15.69:0/3096217692 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7174056440 msgr2=0x7f71740cb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.050+0000 7f71832d6640 1 --2- 172.21.15.69:0/3096217692 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7174056440 0x7f71740cb090 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f716c00b420 tx=0x7f716c00b8e0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.050+0000 7f71832d6640 1 -- 172.21.15.69:0/3096217692 shutdown_connections 2024-04-25T10:59:25.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.050+0000 7f71832d6640 1 --2- 172.21.15.69:0/3096217692 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f7160074530 0x7f7160076a40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.050+0000 7f71832d6640 1 --2- 172.21.15.69:0/3096217692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71740e5b10 0x7f717404d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.050+0000 7f71832d6640 1 --2- 172.21.15.69:0/3096217692 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7174056440 0x7f71740cb090 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.050+0000 7f71832d6640 1 --2- 172.21.15.69:0/3096217692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f717400a4b0 0x7f7174007d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.050+0000 7f71832d6640 1 -- 172.21.15.69:0/3096217692 >> 172.21.15.69:0/3096217692 conn(0x7f717400c3f0 msgr2=0x7f71740d8050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:25.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.050+0000 7f71832d6640 1 -- 172.21.15.69:0/3096217692 shutdown_connections 2024-04-25T10:59:25.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.050+0000 7f71832d6640 1 -- 172.21.15.69:0/3096217692 wait complete. 2024-04-25T10:59:25.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.052+0000 7faefdb55640 1 -- 172.21.15.69:0/688089409 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7faef00051b0 con 0x7faef0053b40 2024-04-25T10:59:25.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.053+0000 7f99c6dee640 1 -- 172.21.15.69:0/3450251409 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f99a40744b0 msgr2=0x7f99a40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.053+0000 7f99c6dee640 1 --2- 172.21.15.69:0/3450251409 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f99a40744b0 0x7f99a40769c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f99b4007a20 tx=0x7f99b4007bf0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.053+0000 7faef57fa640 1 -- 172.21.15.69:0/688089409 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7faeec0881a0 con 0x7faef0053b40 2024-04-25T10:59:25.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.053+0000 7f99c6dee640 1 -- 172.21.15.69:0/3450251409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99b8050e00 msgr2=0x7f99b8178b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.053+0000 7f99c6dee640 1 --2- 172.21.15.69:0/3450251409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99b8050e00 0x7f99b8178b00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f99ac012be0 tx=0x7f99ac010490 comp rx=0 tx=0).stop 2024-04-25T10:59:25.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.053+0000 7f99c6dee640 1 -- 172.21.15.69:0/3450251409 shutdown_connections 2024-04-25T10:59:25.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.053+0000 7f99c6dee640 1 --2- 172.21.15.69:0/3450251409 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f99a40744b0 0x7f99a40769c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.053+0000 7f99c6dee640 1 --2- 172.21.15.69:0/3450251409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99b8054520 0x7f99b8179040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.053+0000 7f99c6dee640 1 --2- 172.21.15.69:0/3450251409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99b8050e00 0x7f99b8178b00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.053+0000 7f99c6dee640 1 --2- 172.21.15.69:0/3450251409 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99b8039ec0 0x7f99b80d7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.053+0000 7f99c6dee640 1 -- 172.21.15.69:0/3450251409 >> 172.21.15.69:0/3450251409 conn(0x7f99b8055e30 msgr2=0x7f99b80e1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:25.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.053+0000 7f99c6dee640 1 -- 172.21.15.69:0/3450251409 shutdown_connections 2024-04-25T10:59:25.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.053+0000 7f99c6dee640 1 -- 172.21.15.69:0/3450251409 wait complete. 2024-04-25T10:59:25.057 INFO:teuthology.orchestra.run.smithi069.stdout:55834574855 2024-04-25T10:59:25.058 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.3 2024-04-25T10:59:25.058 DEBUG:teuthology.parallel:result is None 2024-04-25T10:59:25.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.057+0000 7faefdb55640 1 -- 172.21.15.69:0/688089409 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7faedc0744b0 msgr2=0x7faedc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.057+0000 7faefdb55640 1 --2- 172.21.15.69:0/688089409 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7faedc0744b0 0x7faedc0769c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7faef806bc20 tx=0x7faef806a5c0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.057+0000 7faefdb55640 1 -- 172.21.15.69:0/688089409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faef0053b40 msgr2=0x7faef00e20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.057+0000 7faefdb55640 1 --2- 172.21.15.69:0/688089409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faef0053b40 0x7faef00e20d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7faeec031000 tx=0x7faeec0046c0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.057+0000 7faefdb55640 1 -- 172.21.15.69:0/688089409 shutdown_connections 2024-04-25T10:59:25.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.057+0000 7faefdb55640 1 --2- 172.21.15.69:0/688089409 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7faedc0744b0 0x7faedc0769c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.057+0000 7faefdb55640 1 --2- 172.21.15.69:0/688089409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faef0053b40 0x7faef00e20d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.057+0000 7faefdb55640 1 --2- 172.21.15.69:0/688089409 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faef0048300 0x7faef004cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.057+0000 7faefdb55640 1 --2- 172.21.15.69:0/688089409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faef0047db0 0x7faef00e4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.057+0000 7faefdb55640 1 -- 172.21.15.69:0/688089409 >> 172.21.15.69:0/688089409 conn(0x7faef0054cf0 msgr2=0x7faef0055100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:25.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.057+0000 7faefdb55640 1 -- 172.21.15.69:0/688089409 shutdown_connections 2024-04-25T10:59:25.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.057+0000 7faefdb55640 1 -- 172.21.15.69:0/688089409 wait complete. 2024-04-25T10:59:25.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.059+0000 7f80bf59e640 1 -- 172.21.15.69:0/691542650 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f80a4005850 con 0x7f80c00b46d0 2024-04-25T10:59:25.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.060+0000 7f80aeffd640 1 -- 172.21.15.69:0/691542650 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f80b005ffa0 con 0x7f80c00b46d0 2024-04-25T10:59:25.061 INFO:teuthology.orchestra.run.smithi069.stdout:55834574855 2024-04-25T10:59:25.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.064+0000 7f80bf59e640 1 -- 172.21.15.69:0/691542650 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f80a0074500 msgr2=0x7f80a0076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.064+0000 7f80bf59e640 1 --2- 172.21.15.69:0/691542650 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f80a0074500 0x7f80a0076a10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f80b4034000 tx=0x7f80b402e000 comp rx=0 tx=0).stop 2024-04-25T10:59:25.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.064+0000 7f80bf59e640 1 -- 172.21.15.69:0/691542650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80c00b46d0 msgr2=0x7f80c01604a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.064+0000 7f80bf59e640 1 --2- 172.21.15.69:0/691542650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80c00b46d0 0x7f80c01604a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f80b000b410 tx=0x7f80b000b8d0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.064+0000 7f80bf59e640 1 -- 172.21.15.69:0/691542650 shutdown_connections 2024-04-25T10:59:25.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.064+0000 7f80bf59e640 1 --2- 172.21.15.69:0/691542650 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f80a0074500 0x7f80a0076a10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.064+0000 7f80bf59e640 1 --2- 172.21.15.69:0/691542650 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80c00d4e90 0x7f80c01fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.064+0000 7f80bf59e640 1 --2- 172.21.15.69:0/691542650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80c00b75b0 0x7f80c01fe2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.064+0000 7f80bf59e640 1 --2- 172.21.15.69:0/691542650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80c00b46d0 0x7f80c01604a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.064+0000 7f80bf59e640 1 -- 172.21.15.69:0/691542650 >> 172.21.15.69:0/691542650 conn(0x7f80c00d25b0 msgr2=0x7f80c00b6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:25.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.065+0000 7f80bf59e640 1 -- 172.21.15.69:0/691542650 shutdown_connections 2024-04-25T10:59:25.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.065+0000 7f80bf59e640 1 -- 172.21.15.69:0/691542650 wait complete. 2024-04-25T10:59:25.067 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2024-04-25T10:59:25.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.067+0000 7f3fb62e0640 1 -- 172.21.15.69:0/1935169836 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f3fa8005570 con 0x7f3fa8047db0 2024-04-25T10:59:25.068 DEBUG:teuthology.parallel:result is None 2024-04-25T10:59:25.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.067+0000 7f3facff9640 1 -- 172.21.15.69:0/1935169836 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3fa404b020 con 0x7f3fa8047db0 2024-04-25T10:59:25.068 INFO:teuthology.orchestra.run.smithi069.stdout:55834574856 2024-04-25T10:59:25.069 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.4 2024-04-25T10:59:25.069 DEBUG:teuthology.parallel:result is None 2024-04-25T10:59:25.071 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2024-04-25T10:59:25.071 DEBUG:teuthology.parallel:result is None 2024-04-25T10:59:25.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.070+0000 7f3fb62e0640 1 -- 172.21.15.69:0/1935169836 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3f94074580 msgr2=0x7f3f94076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.070+0000 7f3fb62e0640 1 --2- 172.21.15.69:0/1935169836 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3f94074580 0x7f3f94076a90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3fa80e1660 tx=0x7f3f9c006b50 comp rx=0 tx=0).stop 2024-04-25T10:59:25.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.070+0000 7f3fb62e0640 1 -- 172.21.15.69:0/1935169836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fa8047db0 msgr2=0x7f3fa80df950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.070+0000 7f3fb62e0640 1 --2- 172.21.15.69:0/1935169836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fa8047db0 0x7f3fa80df950 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3fa402aa40 tx=0x7f3fa40046c0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.071+0000 7f3fb62e0640 1 -- 172.21.15.69:0/1935169836 shutdown_connections 2024-04-25T10:59:25.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.071+0000 7f3fb62e0640 1 --2- 172.21.15.69:0/1935169836 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3f94074580 0x7f3f94076a90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.071+0000 7f3fb62e0640 1 --2- 172.21.15.69:0/1935169836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fa804c770 0x7f3fa804de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.071+0000 7f3fb62e0640 1 --2- 172.21.15.69:0/1935169836 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3fa8048300 0x7f3fa804d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.071+0000 7f3fb62e0640 1 --2- 172.21.15.69:0/1935169836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fa8047db0 0x7f3fa80df950 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.071+0000 7f3fb62e0640 1 -- 172.21.15.69:0/1935169836 >> 172.21.15.69:0/1935169836 conn(0x7f3fa8054cf0 msgr2=0x7f3fa8006f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:25.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.071+0000 7f3fb62e0640 1 -- 172.21.15.69:0/1935169836 shutdown_connections 2024-04-25T10:59:25.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.071+0000 7f3fb62e0640 1 -- 172.21.15.69:0/1935169836 wait complete. 2024-04-25T10:59:25.077 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2024-04-25T10:59:25.077 DEBUG:teuthology.parallel:result is None 2024-04-25T10:59:25.080 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-04-25T10:59:25.080 DEBUG:teuthology.parallel:result is None 2024-04-25T10:59:25.084 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.6 2024-04-25T10:59:25.084 DEBUG:teuthology.parallel:result is None 2024-04-25T10:59:25.084 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T10:59:25.085 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:59:25.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.230+0000 7fe33c884640 1 Processor -- start 2024-04-25T10:59:25.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.230+0000 7fe33c884640 1 -- start start 2024-04-25T10:59:25.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.231+0000 7fe33c884640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe338153820 0x7fe3380b6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.231+0000 7fe33c884640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3380cf510 con 0x7fe3380d0bb0 2024-04-25T10:59:25.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.231+0000 7fe33c884640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3380b5380 con 0x7fe33815d960 2024-04-25T10:59:25.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.231+0000 7fe33c884640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fe3380b5660 con 0x7fe338153820 2024-04-25T10:59:25.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.231+0000 7fe32ffff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe338153820 0x7fe3380b6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.232+0000 7fe32ffff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe338153820 0x7fe3380b6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41488/0 (socket says 172.21.15.69:41488) 2024-04-25T10:59:25.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.232+0000 7fe32ffff640 1 -- 172.21.15.69:0/3711857006 learned_addr learned my addr 172.21.15.69:0/3711857006 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:25.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.231+0000 7fe336d9d640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fe3380d0bb0 0x7fe3380d0fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:57226/0 (socket says 172.21.15.69:57226) 2024-04-25T10:59:25.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.232+0000 7fe32ffff640 1 -- 172.21.15.69:0/3711857006 >> v1:172.21.15.146:6789/0 conn(0x7fe33815d960 legacy=0x7fe3380cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.232+0000 7fe335d9b640 1 -- 172.21.15.69:0/3711857006 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2557718624 0 0) 0x7fe3380cf510 con 0x7fe3380d0bb0 2024-04-25T10:59:25.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.232+0000 7fe32ffff640 1 -- 172.21.15.69:0/3711857006 >> v1:172.21.15.69:6789/0 conn(0x7fe3380d0bb0 legacy=0x7fe3380d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.232+0000 7fe32ffff640 1 -- 172.21.15.69:0/3711857006 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3380b5940 con 0x7fe338153820 2024-04-25T10:59:25.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.232+0000 7fe32ffff640 1 --2- 172.21.15.69:0/3711857006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe338153820 0x7fe3380b6140 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe32000aa10 tx=0x7fe32002b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb9dc6aa986401d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:25.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.233+0000 7fe335d9b640 1 -- 172.21.15.69:0/3711857006 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32000b040 con 0x7fe338153820 2024-04-25T10:59:25.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.233+0000 7fe335d9b640 1 -- 172.21.15.69:0/3711857006 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe32002be50 con 0x7fe338153820 2024-04-25T10:59:25.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.233+0000 7fe335d9b640 1 -- 172.21.15.69:0/3711857006 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe320030930 con 0x7fe338153820 2024-04-25T10:59:25.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.233+0000 7fe33c884640 1 -- 172.21.15.69:0/3711857006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe338153820 msgr2=0x7fe3380b6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.233+0000 7fe33c884640 1 --2- 172.21.15.69:0/3711857006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe338153820 0x7fe3380b6140 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe32000aa10 tx=0x7fe32002b630 comp rx=0 tx=0).stop 2024-04-25T10:59:25.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.233+0000 7fe33c884640 1 -- 172.21.15.69:0/3711857006 shutdown_connections 2024-04-25T10:59:25.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.233+0000 7fe33c884640 1 --2- 172.21.15.69:0/3711857006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe338153820 0x7fe3380b6140 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.233+0000 7fe33c884640 1 -- 172.21.15.69:0/3711857006 >> 172.21.15.69:0/3711857006 conn(0x7fe338073050 msgr2=0x7fe338073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:25.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.233+0000 7fe33c884640 1 -- 172.21.15.69:0/3711857006 shutdown_connections 2024-04-25T10:59:25.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.233+0000 7fe33c884640 1 -- 172.21.15.69:0/3711857006 wait complete. 2024-04-25T10:59:25.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.235+0000 7fe33c884640 1 Processor -- start 2024-04-25T10:59:25.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.235+0000 7fe33c884640 1 -- start start 2024-04-25T10:59:25.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.237+0000 7fe33c884640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3380d0bb0 0x7fe338141120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.237+0000 7fe33c884640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe338153820 0x7fe33815b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.237+0000 7fe33c884640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe33815d960 0x7fe33814fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.237+0000 7fe33c884640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe33806d750 con 0x7fe338153820 2024-04-25T10:59:25.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.237+0000 7fe33c884640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe33806d5d0 con 0x7fe33815d960 2024-04-25T10:59:25.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.237+0000 7fe33c884640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe33806d8d0 con 0x7fe3380d0bb0 2024-04-25T10:59:25.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.237+0000 7fe32ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe338153820 0x7fe33815b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.237+0000 7fe32ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe338153820 0x7fe33815b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41502/0 (socket says 172.21.15.69:41502) 2024-04-25T10:59:25.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.237+0000 7fe32ffff640 1 -- 172.21.15.69:0/3637204694 learned_addr learned my addr 172.21.15.69:0/3637204694 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:25.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.237+0000 7fe336d9d640 1 --2- 172.21.15.69:0/3637204694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe33815d960 0x7fe33814fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.237+0000 7fe33659c640 1 --2- 172.21.15.69:0/3637204694 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3380d0bb0 0x7fe338141120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.237+0000 7fe32ffff640 1 -- 172.21.15.69:0/3637204694 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3380d0bb0 msgr2=0x7fe338141120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.237+0000 7fe32ffff640 1 --2- 172.21.15.69:0/3637204694 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3380d0bb0 0x7fe338141120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.237+0000 7fe32ffff640 1 -- 172.21.15.69:0/3637204694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe33815d960 msgr2=0x7fe33814fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.237+0000 7fe32ffff640 1 --2- 172.21.15.69:0/3637204694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe33815d960 0x7fe33814fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.237+0000 7fe32ffff640 1 -- 172.21.15.69:0/3637204694 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe33815c490 con 0x7fe338153820 2024-04-25T10:59:25.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.237+0000 7fe32ffff640 1 --2- 172.21.15.69:0/3637204694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe338153820 0x7fe33815b7a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe320000c00 tx=0x7fe3200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:25.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.237+0000 7fe32effd640 1 -- 172.21.15.69:0/3637204694 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32003f020 con 0x7fe338153820 2024-04-25T10:59:25.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.237+0000 7fe32effd640 1 -- 172.21.15.69:0/3637204694 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe320032990 con 0x7fe338153820 2024-04-25T10:59:25.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.237+0000 7fe33c884640 1 -- 172.21.15.69:0/3637204694 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe33815c770 con 0x7fe338153820 2024-04-25T10:59:25.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.237+0000 7fe32effd640 1 -- 172.21.15.69:0/3637204694 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe320032d40 con 0x7fe338153820 2024-04-25T10:59:25.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.237+0000 7fe33c884640 1 -- 172.21.15.69:0/3637204694 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe33815cd30 con 0x7fe338153820 2024-04-25T10:59:25.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.238+0000 7fe33c884640 1 -- 172.21.15.69:0/3637204694 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe338050740 con 0x7fe338153820 2024-04-25T10:59:25.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.239+0000 7fe32effd640 1 -- 172.21.15.69:0/3637204694 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe320042020 con 0x7fe338153820 2024-04-25T10:59:25.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.239+0000 7fe32effd640 1 --2- 172.21.15.69:0/3637204694 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe3180744b0 0x7fe3180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.240+0000 7fe33659c640 1 --2- 172.21.15.69:0/3637204694 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe3180744b0 0x7fe3180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.240+0000 7fe32effd640 1 -- 172.21.15.69:0/3637204694 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe3200b69e0 con 0x7fe338153820 2024-04-25T10:59:25.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.240+0000 7fe33659c640 1 --2- 172.21.15.69:0/3637204694 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe3180744b0 0x7fe3180769c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe328000f60 tx=0x7fe328009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:25.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.244+0000 7fe32effd640 1 -- 172.21.15.69:0/3637204694 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe320049030 con 0x7fe338153820 2024-04-25T10:59:25.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.416+0000 7fe33c884640 1 -- 172.21.15.69:0/3637204694 --> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe33815c220 con 0x7fe3180744b0 2024-04-25T10:59:25.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.422+0000 7fe32effd640 1 -- 172.21.15.69:0/3637204694 <== mgr.4107 v2:172.21.15.69:6832/1107421267 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251825 (secure 0 0 0) 0x7fe33815c220 con 0x7fe3180744b0 2024-04-25T10:59:25.425 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:25.426 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-04-25T10:59:25.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.425+0000 7fe33c884640 1 -- 172.21.15.69:0/3637204694 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe3180744b0 msgr2=0x7fe3180769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.425+0000 7fe33c884640 1 --2- 172.21.15.69:0/3637204694 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe3180744b0 0x7fe3180769c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe328000f60 tx=0x7fe328009040 comp rx=0 tx=0).stop 2024-04-25T10:59:25.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.425+0000 7fe33c884640 1 -- 172.21.15.69:0/3637204694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe338153820 msgr2=0x7fe33815b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.425+0000 7fe33c884640 1 --2- 172.21.15.69:0/3637204694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe338153820 0x7fe33815b7a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe320000c00 tx=0x7fe3200046c0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.425+0000 7fe33c884640 1 -- 172.21.15.69:0/3637204694 shutdown_connections 2024-04-25T10:59:25.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.425+0000 7fe33c884640 1 --2- 172.21.15.69:0/3637204694 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe3180744b0 0x7fe3180769c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.425+0000 7fe33c884640 1 --2- 172.21.15.69:0/3637204694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe33815d960 0x7fe33814fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.425+0000 7fe33c884640 1 --2- 172.21.15.69:0/3637204694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe338153820 0x7fe33815b7a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.425+0000 7fe33c884640 1 --2- 172.21.15.69:0/3637204694 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3380d0bb0 0x7fe338141120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.425+0000 7fe33c884640 1 -- 172.21.15.69:0/3637204694 >> 172.21.15.69:0/3637204694 conn(0x7fe338073050 msgr2=0x7fe33815e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.425+0000 7fe33c884640 1 -- 172.21.15.69:0/3637204694 shutdown_connections 2024-04-25T10:59:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.425+0000 7fe33c884640 1 -- 172.21.15.69:0/3637204694 wait complete. 2024-04-25T10:59:25.437 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-04-25T10:59:24.406977+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219012,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755708,"statfs":{"total":773094113280,"available":772869844992,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001259"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228976+0000","last_change":"2024-04-25T10:59:19.640957+0000","last_active":"2024-04-25T10:59:21.228976+0000","last_peered":"2024-04-25T10:59:21.228976+0000","last_clean":"2024-04-25T10:59:21.228976+0000","last_became_active":"2024-04-25T10:59:19.640610+0000","last_became_peered":"2024-04-25T10:59:19.640610+0000","last_unstale":"2024-04-25T10:59:21.228976+0000","last_undegraded":"2024-04-25T10:59:21.228976+0000","last_fullsized":"2024-04-25T10:59:21.228976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:42.867755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636604+0000","last_change":"2024-04-25T10:59:19.644473+0000","last_active":"2024-04-25T10:59:20.636604+0000","last_peered":"2024-04-25T10:59:20.636604+0000","last_clean":"2024-04-25T10:59:20.636604+0000","last_became_active":"2024-04-25T10:59:19.644151+0000","last_became_peered":"2024-04-25T10:59:19.644151+0000","last_unstale":"2024-04-25T10:59:20.636604+0000","last_undegraded":"2024-04-25T10:59:20.636604+0000","last_fullsized":"2024-04-25T10:59:20.636604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:07.522883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227761+0000","last_change":"2024-04-25T10:59:19.630984+0000","last_active":"2024-04-25T10:59:21.227761+0000","last_peered":"2024-04-25T10:59:21.227761+0000","last_clean":"2024-04-25T10:59:21.227761+0000","last_became_active":"2024-04-25T10:59:19.630699+0000","last_became_peered":"2024-04-25T10:59:19.630699+0000","last_unstale":"2024-04-25T10:59:21.227761+0000","last_undegraded":"2024-04-25T10:59:21.227761+0000","last_fullsized":"2024-04-25T10:59:21.227761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:59.213210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228437+0000","last_change":"2024-04-25T10:59:19.649708+0000","last_active":"2024-04-25T10:59:21.228437+0000","last_peered":"2024-04-25T10:59:21.228437+0000","last_clean":"2024-04-25T10:59:21.228437+0000","last_became_active":"2024-04-25T10:59:19.649262+0000","last_became_peered":"2024-04-25T10:59:19.649262+0000","last_unstale":"2024-04-25T10:59:21.228437+0000","last_undegraded":"2024-04-25T10:59:21.228437+0000","last_fullsized":"2024-04-25T10:59:21.228437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:52.334437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228519+0000","last_change":"2024-04-25T10:59:19.642490+0000","last_active":"2024-04-25T10:59:21.228519+0000","last_peered":"2024-04-25T10:59:21.228519+0000","last_clean":"2024-04-25T10:59:21.228519+0000","last_became_active":"2024-04-25T10:59:19.642122+0000","last_became_peered":"2024-04-25T10:59:19.642122+0000","last_unstale":"2024-04-25T10:59:21.228519+0000","last_undegraded":"2024-04-25T10:59:21.228519+0000","last_fullsized":"2024-04-25T10:59:21.228519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:14.189727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228892+0000","last_change":"2024-04-25T10:59:19.651061+0000","last_active":"2024-04-25T10:59:21.228892+0000","last_peered":"2024-04-25T10:59:21.228892+0000","last_clean":"2024-04-25T10:59:21.228892+0000","last_became_active":"2024-04-25T10:59:19.650710+0000","last_became_peered":"2024-04-25T10:59:19.650710+0000","last_unstale":"2024-04-25T10:59:21.228892+0000","last_undegraded":"2024-04-25T10:59:21.228892+0000","last_fullsized":"2024-04-25T10:59:21.228892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:45.328749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227817+0000","last_change":"2024-04-25T10:59:19.646862+0000","last_active":"2024-04-25T10:59:21.227817+0000","last_peered":"2024-04-25T10:59:21.227817+0000","last_clean":"2024-04-25T10:59:21.227817+0000","last_became_active":"2024-04-25T10:59:19.646553+0000","last_became_peered":"2024-04-25T10:59:19.646553+0000","last_unstale":"2024-04-25T10:59:21.227817+0000","last_undegraded":"2024-04-25T10:59:21.227817+0000","last_fullsized":"2024-04-25T10:59:21.227817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:52.334437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227508+0000","last_change":"2024-04-25T10:59:19.648086+0000","last_active":"2024-04-25T10:59:21.227508+0000","last_peered":"2024-04-25T10:59:21.227508+0000","last_clean":"2024-04-25T10:59:21.227508+0000","last_became_active":"2024-04-25T10:59:19.647702+0000","last_became_peered":"2024-04-25T10:59:19.647702+0000","last_unstale":"2024-04-25T10:59:21.227508+0000","last_undegraded":"2024-04-25T10:59:21.227508+0000","last_fullsized":"2024-04-25T10:59:21.227508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:59.213210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228983+0000","last_change":"2024-04-25T10:59:19.655202+0000","last_active":"2024-04-25T10:59:21.228983+0000","last_peered":"2024-04-25T10:59:21.228983+0000","last_clean":"2024-04-25T10:59:21.228983+0000","last_became_active":"2024-04-25T10:59:19.654871+0000","last_became_peered":"2024-04-25T10:59:19.654871+0000","last_unstale":"2024-04-25T10:59:21.228983+0000","last_undegraded":"2024-04-25T10:59:21.228983+0000","last_fullsized":"2024-04-25T10:59:21.228983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:50.893634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638094+0000","last_change":"2024-04-25T10:59:19.651928+0000","last_active":"2024-04-25T10:59:20.638094+0000","last_peered":"2024-04-25T10:59:20.638094+0000","last_clean":"2024-04-25T10:59:20.638094+0000","last_became_active":"2024-04-25T10:59:19.651618+0000","last_became_peered":"2024-04-25T10:59:19.651618+0000","last_unstale":"2024-04-25T10:59:20.638094+0000","last_undegraded":"2024-04-25T10:59:20.638094+0000","last_fullsized":"2024-04-25T10:59:20.638094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:32:26.553467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638187+0000","last_change":"2024-04-25T10:59:19.652810+0000","last_active":"2024-04-25T10:59:20.638187+0000","last_peered":"2024-04-25T10:59:20.638187+0000","last_clean":"2024-04-25T10:59:20.638187+0000","last_became_active":"2024-04-25T10:59:19.652118+0000","last_became_peered":"2024-04-25T10:59:19.652118+0000","last_unstale":"2024-04-25T10:59:20.638187+0000","last_undegraded":"2024-04-25T10:59:20.638187+0000","last_fullsized":"2024-04-25T10:59:20.638187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:59.213210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638421+0000","last_change":"2024-04-25T10:59:19.645548+0000","last_active":"2024-04-25T10:59:20.638421+0000","last_peered":"2024-04-25T10:59:20.638421+0000","last_clean":"2024-04-25T10:59:20.638421+0000","last_became_active":"2024-04-25T10:59:19.645231+0000","last_became_peered":"2024-04-25T10:59:19.645231+0000","last_unstale":"2024-04-25T10:59:20.638421+0000","last_undegraded":"2024-04-25T10:59:20.638421+0000","last_fullsized":"2024-04-25T10:59:20.638421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:59.512598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227723+0000","last_change":"2024-04-25T10:59:19.640736+0000","last_active":"2024-04-25T10:59:21.227723+0000","last_peered":"2024-04-25T10:59:21.227723+0000","last_clean":"2024-04-25T10:59:21.227723+0000","last_became_active":"2024-04-25T10:59:19.640448+0000","last_became_peered":"2024-04-25T10:59:19.640448+0000","last_unstale":"2024-04-25T10:59:21.227723+0000","last_undegraded":"2024-04-25T10:59:21.227723+0000","last_fullsized":"2024-04-25T10:59:21.227723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:15.132862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228215+0000","last_change":"2024-04-25T10:59:19.634929+0000","last_active":"2024-04-25T10:59:21.228215+0000","last_peered":"2024-04-25T10:59:21.228215+0000","last_clean":"2024-04-25T10:59:21.228215+0000","last_became_active":"2024-04-25T10:59:19.634640+0000","last_became_peered":"2024-04-25T10:59:19.634640+0000","last_unstale":"2024-04-25T10:59:21.228215+0000","last_undegraded":"2024-04-25T10:59:21.228215+0000","last_fullsized":"2024-04-25T10:59:21.228215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:49.314956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636375+0000","last_change":"2024-04-25T10:59:19.648597+0000","last_active":"2024-04-25T10:59:20.636375+0000","last_peered":"2024-04-25T10:59:20.636375+0000","last_clean":"2024-04-25T10:59:20.636375+0000","last_became_active":"2024-04-25T10:59:19.648280+0000","last_became_peered":"2024-04-25T10:59:19.648280+0000","last_unstale":"2024-04-25T10:59:20.636375+0000","last_undegraded":"2024-04-25T10:59:20.636375+0000","last_fullsized":"2024-04-25T10:59:20.636375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:58.824966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228034+0000","last_change":"2024-04-25T10:59:19.647183+0000","last_active":"2024-04-25T10:59:21.228034+0000","last_peered":"2024-04-25T10:59:21.228034+0000","last_clean":"2024-04-25T10:59:21.228034+0000","last_became_active":"2024-04-25T10:59:19.646563+0000","last_became_peered":"2024-04-25T10:59:19.646563+0000","last_unstale":"2024-04-25T10:59:21.228034+0000","last_undegraded":"2024-04-25T10:59:21.228034+0000","last_fullsized":"2024-04-25T10:59:21.228034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:57.344787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228532+0000","last_change":"2024-04-25T10:59:19.649207+0000","last_active":"2024-04-25T10:59:21.228532+0000","last_peered":"2024-04-25T10:59:21.228532+0000","last_clean":"2024-04-25T10:59:21.228532+0000","last_became_active":"2024-04-25T10:59:19.647280+0000","last_became_peered":"2024-04-25T10:59:19.647280+0000","last_unstale":"2024-04-25T10:59:21.228532+0000","last_undegraded":"2024-04-25T10:59:21.228532+0000","last_fullsized":"2024-04-25T10:59:21.228532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:32.457404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.185793+0000","last_change":"2024-04-25T10:59:19.640810+0000","last_active":"2024-04-25T10:59:21.185793+0000","last_peered":"2024-04-25T10:59:21.185793+0000","last_clean":"2024-04-25T10:59:21.185793+0000","last_became_active":"2024-04-25T10:59:19.640524+0000","last_became_peered":"2024-04-25T10:59:19.640524+0000","last_unstale":"2024-04-25T10:59:21.185793+0000","last_undegraded":"2024-04-25T10:59:21.185793+0000","last_fullsized":"2024-04-25T10:59:21.185793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:58.824966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228703+0000","last_change":"2024-04-25T10:59:19.641698+0000","last_active":"2024-04-25T10:59:21.228703+0000","last_peered":"2024-04-25T10:59:21.228703+0000","last_clean":"2024-04-25T10:59:21.228703+0000","last_became_active":"2024-04-25T10:59:19.640996+0000","last_became_peered":"2024-04-25T10:59:19.640996+0000","last_unstale":"2024-04-25T10:59:21.228703+0000","last_undegraded":"2024-04-25T10:59:21.228703+0000","last_fullsized":"2024-04-25T10:59:21.228703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:33.154184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.639226+0000","last_change":"2024-04-25T10:59:19.654176+0000","last_active":"2024-04-25T10:59:20.639226+0000","last_peered":"2024-04-25T10:59:20.639226+0000","last_clean":"2024-04-25T10:59:20.639226+0000","last_became_active":"2024-04-25T10:59:19.653175+0000","last_became_peered":"2024-04-25T10:59:19.653175+0000","last_unstale":"2024-04-25T10:59:20.639226+0000","last_undegraded":"2024-04-25T10:59:20.639226+0000","last_fullsized":"2024-04-25T10:59:20.639226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:33.050282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227935+0000","last_change":"2024-04-25T10:59:19.642959+0000","last_active":"2024-04-25T10:59:21.227935+0000","last_peered":"2024-04-25T10:59:21.227935+0000","last_clean":"2024-04-25T10:59:21.227935+0000","last_became_active":"2024-04-25T10:59:19.642671+0000","last_became_peered":"2024-04-25T10:59:19.642671+0000","last_unstale":"2024-04-25T10:59:21.227935+0000","last_undegraded":"2024-04-25T10:59:21.227935+0000","last_fullsized":"2024-04-25T10:59:21.227935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:15.132862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228503+0000","last_change":"2024-04-25T10:59:19.658039+0000","last_active":"2024-04-25T10:59:21.228503+0000","last_peered":"2024-04-25T10:59:21.228503+0000","last_clean":"2024-04-25T10:59:21.228503+0000","last_became_active":"2024-04-25T10:59:19.657194+0000","last_became_peered":"2024-04-25T10:59:19.657194+0000","last_unstale":"2024-04-25T10:59:21.228503+0000","last_undegraded":"2024-04-25T10:59:21.228503+0000","last_fullsized":"2024-04-25T10:59:21.228503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:19.485287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227782+0000","last_change":"2024-04-25T10:59:19.645451+0000","last_active":"2024-04-25T10:59:21.227782+0000","last_peered":"2024-04-25T10:59:21.227782+0000","last_clean":"2024-04-25T10:59:21.227782+0000","last_became_active":"2024-04-25T10:59:19.645121+0000","last_became_peered":"2024-04-25T10:59:19.645121+0000","last_unstale":"2024-04-25T10:59:21.227782+0000","last_undegraded":"2024-04-25T10:59:21.227782+0000","last_fullsized":"2024-04-25T10:59:21.227782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:07.522883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228328+0000","last_change":"2024-04-25T10:59:19.646916+0000","last_active":"2024-04-25T10:59:21.228328+0000","last_peered":"2024-04-25T10:59:21.228328+0000","last_clean":"2024-04-25T10:59:21.228328+0000","last_became_active":"2024-04-25T10:59:19.646228+0000","last_became_peered":"2024-04-25T10:59:19.646228+0000","last_unstale":"2024-04-25T10:59:21.228328+0000","last_undegraded":"2024-04-25T10:59:21.228328+0000","last_fullsized":"2024-04-25T10:59:21.228328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:17.431757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228328+0000","last_change":"2024-04-25T10:59:19.644795+0000","last_active":"2024-04-25T10:59:21.228328+0000","last_peered":"2024-04-25T10:59:21.228328+0000","last_clean":"2024-04-25T10:59:21.228328+0000","last_became_active":"2024-04-25T10:59:19.644503+0000","last_became_peered":"2024-04-25T10:59:19.644503+0000","last_unstale":"2024-04-25T10:59:21.228328+0000","last_undegraded":"2024-04-25T10:59:21.228328+0000","last_fullsized":"2024-04-25T10:59:21.228328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:17.431757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638291+0000","last_change":"2024-04-25T10:59:19.650630+0000","last_active":"2024-04-25T10:59:20.638291+0000","last_peered":"2024-04-25T10:59:20.638291+0000","last_clean":"2024-04-25T10:59:20.638291+0000","last_became_active":"2024-04-25T10:59:19.650312+0000","last_became_peered":"2024-04-25T10:59:19.650312+0000","last_unstale":"2024-04-25T10:59:20.638291+0000","last_undegraded":"2024-04-25T10:59:20.638291+0000","last_fullsized":"2024-04-25T10:59:20.638291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:58.824966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636446+0000","last_change":"2024-04-25T10:59:19.641641+0000","last_active":"2024-04-25T10:59:20.636446+0000","last_peered":"2024-04-25T10:59:20.636446+0000","last_clean":"2024-04-25T10:59:20.636446+0000","last_became_active":"2024-04-25T10:59:19.641339+0000","last_became_peered":"2024-04-25T10:59:19.641339+0000","last_unstale":"2024-04-25T10:59:20.636446+0000","last_undegraded":"2024-04-25T10:59:20.636446+0000","last_fullsized":"2024-04-25T10:59:20.636446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:52.815596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228173+0000","last_change":"2024-04-25T10:59:19.642824+0000","last_active":"2024-04-25T10:59:21.228173+0000","last_peered":"2024-04-25T10:59:21.228173+0000","last_clean":"2024-04-25T10:59:21.228173+0000","last_became_active":"2024-04-25T10:59:19.641152+0000","last_became_peered":"2024-04-25T10:59:19.641152+0000","last_unstale":"2024-04-25T10:59:21.228173+0000","last_undegraded":"2024-04-25T10:59:21.228173+0000","last_fullsized":"2024-04-25T10:59:21.228173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:49.314956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228117+0000","last_change":"2024-04-25T10:59:19.629558+0000","last_active":"2024-04-25T10:59:21.228117+0000","last_peered":"2024-04-25T10:59:21.228117+0000","last_clean":"2024-04-25T10:59:21.228117+0000","last_became_active":"2024-04-25T10:59:19.628864+0000","last_became_peered":"2024-04-25T10:59:19.628864+0000","last_unstale":"2024-04-25T10:59:21.228117+0000","last_undegraded":"2024-04-25T10:59:21.228117+0000","last_fullsized":"2024-04-25T10:59:21.228117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:07.522883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228910+0000","last_change":"2024-04-25T10:59:19.648576+0000","last_active":"2024-04-25T10:59:21.228910+0000","last_peered":"2024-04-25T10:59:21.228910+0000","last_clean":"2024-04-25T10:59:21.228910+0000","last_became_active":"2024-04-25T10:59:19.647435+0000","last_became_peered":"2024-04-25T10:59:19.647435+0000","last_unstale":"2024-04-25T10:59:21.228910+0000","last_undegraded":"2024-04-25T10:59:21.228910+0000","last_fullsized":"2024-04-25T10:59:21.228910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:42.867755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.185603+0000","last_change":"2024-04-25T10:59:19.649031+0000","last_active":"2024-04-25T10:59:21.185603+0000","last_peered":"2024-04-25T10:59:21.185603+0000","last_clean":"2024-04-25T10:59:21.185603+0000","last_became_active":"2024-04-25T10:59:19.648698+0000","last_became_peered":"2024-04-25T10:59:19.648698+0000","last_unstale":"2024-04-25T10:59:21.185603+0000","last_undegraded":"2024-04-25T10:59:21.185603+0000","last_fullsized":"2024-04-25T10:59:21.185603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:59.213210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638942+0000","last_change":"2024-04-25T10:59:19.643547+0000","last_active":"2024-04-25T10:59:20.638942+0000","last_peered":"2024-04-25T10:59:20.638942+0000","last_clean":"2024-04-25T10:59:20.638942+0000","last_became_active":"2024-04-25T10:59:19.642802+0000","last_became_peered":"2024-04-25T10:59:19.642802+0000","last_unstale":"2024-04-25T10:59:20.638942+0000","last_undegraded":"2024-04-25T10:59:20.638942+0000","last_fullsized":"2024-04-25T10:59:20.638942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:15.132862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638161+0000","last_change":"2024-04-25T10:59:19.638090+0000","last_active":"2024-04-25T10:59:20.638161+0000","last_peered":"2024-04-25T10:59:20.638161+0000","last_clean":"2024-04-25T10:59:20.638161+0000","last_became_active":"2024-04-25T10:59:19.637745+0000","last_became_peered":"2024-04-25T10:59:19.637745+0000","last_unstale":"2024-04-25T10:59:20.638161+0000","last_undegraded":"2024-04-25T10:59:20.638161+0000","last_fullsized":"2024-04-25T10:59:20.638161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:25.274583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227963+0000","last_change":"2024-04-25T10:59:18.637294+0000","last_active":"2024-04-25T10:59:21.227963+0000","last_peered":"2024-04-25T10:59:21.227963+0000","last_clean":"2024-04-25T10:59:21.227963+0000","last_became_active":"2024-04-25T10:59:18.636963+0000","last_became_peered":"2024-04-25T10:59:18.636963+0000","last_unstale":"2024-04-25T10:59:21.227963+0000","last_undegraded":"2024-04-25T10:59:21.227963+0000","last_fullsized":"2024-04-25T10:59:21.227963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:18.298759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228183+0000","last_change":"2024-04-25T10:59:19.638014+0000","last_active":"2024-04-25T10:59:21.228183+0000","last_peered":"2024-04-25T10:59:21.228183+0000","last_clean":"2024-04-25T10:59:21.228183+0000","last_became_active":"2024-04-25T10:59:19.637471+0000","last_became_peered":"2024-04-25T10:59:19.637471+0000","last_unstale":"2024-04-25T10:59:21.228183+0000","last_undegraded":"2024-04-25T10:59:21.228183+0000","last_fullsized":"2024-04-25T10:59:21.228183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:33.050282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638880+0000","last_change":"2024-04-25T10:59:18.634982+0000","last_active":"2024-04-25T10:59:20.638880+0000","last_peered":"2024-04-25T10:59:20.638880+0000","last_clean":"2024-04-25T10:59:20.638880+0000","last_became_active":"2024-04-25T10:59:18.634658+0000","last_became_peered":"2024-04-25T10:59:18.634658+0000","last_unstale":"2024-04-25T10:59:20.638880+0000","last_undegraded":"2024-04-25T10:59:20.638880+0000","last_fullsized":"2024-04-25T10:59:20.638880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:35.341389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227940+0000","last_change":"2024-04-25T10:59:18.632617+0000","last_active":"2024-04-25T10:59:21.227940+0000","last_peered":"2024-04-25T10:59:21.227940+0000","last_clean":"2024-04-25T10:59:21.227940+0000","last_became_active":"2024-04-25T10:59:18.632294+0000","last_became_peered":"2024-04-25T10:59:18.632294+0000","last_unstale":"2024-04-25T10:59:21.227940+0000","last_undegraded":"2024-04-25T10:59:21.227940+0000","last_fullsized":"2024-04-25T10:59:21.227940+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:46.279987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227575+0000","last_change":"2024-04-25T10:59:19.632333+0000","last_active":"2024-04-25T10:59:21.227575+0000","last_peered":"2024-04-25T10:59:21.227575+0000","last_clean":"2024-04-25T10:59:21.227575+0000","last_became_active":"2024-04-25T10:59:19.632042+0000","last_became_peered":"2024-04-25T10:59:19.632042+0000","last_unstale":"2024-04-25T10:59:21.227575+0000","last_undegraded":"2024-04-25T10:59:21.227575+0000","last_fullsized":"2024-04-25T10:59:21.227575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:58.824966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228184+0000","last_change":"2024-04-25T10:59:18.638103+0000","last_active":"2024-04-25T10:59:21.228184+0000","last_peered":"2024-04-25T10:59:21.228184+0000","last_clean":"2024-04-25T10:59:21.228184+0000","last_became_active":"2024-04-25T10:59:18.636931+0000","last_became_peered":"2024-04-25T10:59:18.636931+0000","last_unstale":"2024-04-25T10:59:21.228184+0000","last_undegraded":"2024-04-25T10:59:21.228184+0000","last_fullsized":"2024-04-25T10:59:21.228184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:15.754476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636374+0000","last_change":"2024-04-25T10:59:19.633809+0000","last_active":"2024-04-25T10:59:20.636374+0000","last_peered":"2024-04-25T10:59:20.636374+0000","last_clean":"2024-04-25T10:59:20.636374+0000","last_became_active":"2024-04-25T10:59:19.632787+0000","last_became_peered":"2024-04-25T10:59:19.632787+0000","last_unstale":"2024-04-25T10:59:20.636374+0000","last_undegraded":"2024-04-25T10:59:20.636374+0000","last_fullsized":"2024-04-25T10:59:20.636374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:59.213210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638419+0000","last_change":"2024-04-25T10:59:18.641813+0000","last_active":"2024-04-25T10:59:20.638419+0000","last_peered":"2024-04-25T10:59:20.638419+0000","last_clean":"2024-04-25T10:59:20.638419+0000","last_became_active":"2024-04-25T10:59:18.641127+0000","last_became_peered":"2024-04-25T10:59:18.641127+0000","last_unstale":"2024-04-25T10:59:20.638419+0000","last_undegraded":"2024-04-25T10:59:20.638419+0000","last_fullsized":"2024-04-25T10:59:20.638419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:28.594312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228699+0000","last_change":"2024-04-25T10:59:19.645823+0000","last_active":"2024-04-25T10:59:21.228699+0000","last_peered":"2024-04-25T10:59:21.228699+0000","last_clean":"2024-04-25T10:59:21.228699+0000","last_became_active":"2024-04-25T10:59:19.645500+0000","last_became_peered":"2024-04-25T10:59:19.645500+0000","last_unstale":"2024-04-25T10:59:21.228699+0000","last_undegraded":"2024-04-25T10:59:21.228699+0000","last_fullsized":"2024-04-25T10:59:21.228699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:00.311029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228172+0000","last_change":"2024-04-25T10:59:18.641002+0000","last_active":"2024-04-25T10:59:21.228172+0000","last_peered":"2024-04-25T10:59:21.228172+0000","last_clean":"2024-04-25T10:59:21.228172+0000","last_became_active":"2024-04-25T10:59:18.640589+0000","last_became_peered":"2024-04-25T10:59:18.640589+0000","last_unstale":"2024-04-25T10:59:21.228172+0000","last_undegraded":"2024-04-25T10:59:21.228172+0000","last_fullsized":"2024-04-25T10:59:21.228172+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:24.267022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227906+0000","last_change":"2024-04-25T10:59:19.652033+0000","last_active":"2024-04-25T10:59:21.227906+0000","last_peered":"2024-04-25T10:59:21.227906+0000","last_clean":"2024-04-25T10:59:21.227906+0000","last_became_active":"2024-04-25T10:59:19.651694+0000","last_became_peered":"2024-04-25T10:59:19.651694+0000","last_unstale":"2024-04-25T10:59:21.227906+0000","last_undegraded":"2024-04-25T10:59:21.227906+0000","last_fullsized":"2024-04-25T10:59:21.227906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:58.824966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636485+0000","last_change":"2024-04-25T10:59:18.638096+0000","last_active":"2024-04-25T10:59:20.636485+0000","last_peered":"2024-04-25T10:59:20.636485+0000","last_clean":"2024-04-25T10:59:20.636485+0000","last_became_active":"2024-04-25T10:59:18.637808+0000","last_became_peered":"2024-04-25T10:59:18.637808+0000","last_unstale":"2024-04-25T10:59:20.636485+0000","last_undegraded":"2024-04-25T10:59:20.636485+0000","last_fullsized":"2024-04-25T10:59:20.636485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:10.473815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638613+0000","last_change":"2024-04-25T10:59:19.638709+0000","last_active":"2024-04-25T10:59:20.638613+0000","last_peered":"2024-04-25T10:59:20.638613+0000","last_clean":"2024-04-25T10:59:20.638613+0000","last_became_active":"2024-04-25T10:59:19.638358+0000","last_became_peered":"2024-04-25T10:59:19.638358+0000","last_unstale":"2024-04-25T10:59:20.638613+0000","last_undegraded":"2024-04-25T10:59:20.638613+0000","last_fullsized":"2024-04-25T10:59:20.638613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:46.619782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.639204+0000","last_change":"2024-04-25T10:59:18.639824+0000","last_active":"2024-04-25T10:59:20.639204+0000","last_peered":"2024-04-25T10:59:20.639204+0000","last_clean":"2024-04-25T10:59:20.639204+0000","last_became_active":"2024-04-25T10:59:18.639316+0000","last_became_peered":"2024-04-25T10:59:18.639316+0000","last_unstale":"2024-04-25T10:59:20.639204+0000","last_undegraded":"2024-04-25T10:59:20.639204+0000","last_fullsized":"2024-04-25T10:59:20.639204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:15.754476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228064+0000","last_change":"2024-04-25T10:59:19.641380+0000","last_active":"2024-04-25T10:59:21.228064+0000","last_peered":"2024-04-25T10:59:21.228064+0000","last_clean":"2024-04-25T10:59:21.228064+0000","last_became_active":"2024-04-25T10:59:19.640580+0000","last_became_peered":"2024-04-25T10:59:19.640580+0000","last_unstale":"2024-04-25T10:59:21.228064+0000","last_undegraded":"2024-04-25T10:59:21.228064+0000","last_fullsized":"2024-04-25T10:59:21.228064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:19.485287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228226+0000","last_change":"2024-04-25T10:59:18.634324+0000","last_active":"2024-04-25T10:59:21.228226+0000","last_peered":"2024-04-25T10:59:21.228226+0000","last_clean":"2024-04-25T10:59:21.228226+0000","last_became_active":"2024-04-25T10:59:18.634049+0000","last_became_peered":"2024-04-25T10:59:18.634049+0000","last_unstale":"2024-04-25T10:59:21.228226+0000","last_undegraded":"2024-04-25T10:59:21.228226+0000","last_fullsized":"2024-04-25T10:59:21.228226+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:02.619679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228376+0000","last_change":"2024-04-25T10:59:19.643539+0000","last_active":"2024-04-25T10:59:21.228376+0000","last_peered":"2024-04-25T10:59:21.228376+0000","last_clean":"2024-04-25T10:59:21.228376+0000","last_became_active":"2024-04-25T10:59:19.643182+0000","last_became_peered":"2024-04-25T10:59:19.643182+0000","last_unstale":"2024-04-25T10:59:21.228376+0000","last_undegraded":"2024-04-25T10:59:21.228376+0000","last_fullsized":"2024-04-25T10:59:21.228376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:57.344787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.229036+0000","last_change":"2024-04-25T10:59:18.643936+0000","last_active":"2024-04-25T10:59:21.229036+0000","last_peered":"2024-04-25T10:59:21.229036+0000","last_clean":"2024-04-25T10:59:21.229036+0000","last_became_active":"2024-04-25T10:59:18.643576+0000","last_became_peered":"2024-04-25T10:59:18.643576+0000","last_unstale":"2024-04-25T10:59:21.229036+0000","last_undegraded":"2024-04-25T10:59:21.229036+0000","last_fullsized":"2024-04-25T10:59:21.229036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:58.505038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228718+0000","last_change":"2024-04-25T10:59:19.652528+0000","last_active":"2024-04-25T10:59:21.228718+0000","last_peered":"2024-04-25T10:59:21.228718+0000","last_clean":"2024-04-25T10:59:21.228718+0000","last_became_active":"2024-04-25T10:59:19.651277+0000","last_became_peered":"2024-04-25T10:59:19.651277+0000","last_unstale":"2024-04-25T10:59:21.228718+0000","last_undegraded":"2024-04-25T10:59:21.228718+0000","last_fullsized":"2024-04-25T10:59:21.228718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:45.328749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.637715+0000","last_change":"2024-04-25T10:59:18.630832+0000","last_active":"2024-04-25T10:59:20.637715+0000","last_peered":"2024-04-25T10:59:20.637715+0000","last_clean":"2024-04-25T10:59:20.637715+0000","last_became_active":"2024-04-25T10:59:18.630462+0000","last_became_peered":"2024-04-25T10:59:18.630462+0000","last_unstale":"2024-04-25T10:59:20.637715+0000","last_undegraded":"2024-04-25T10:59:20.637715+0000","last_fullsized":"2024-04-25T10:59:20.637715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:46.279987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.639078+0000","last_change":"2024-04-25T10:59:19.643805+0000","last_active":"2024-04-25T10:59:20.639078+0000","last_peered":"2024-04-25T10:59:20.639078+0000","last_clean":"2024-04-25T10:59:20.639078+0000","last_became_active":"2024-04-25T10:59:19.643222+0000","last_became_peered":"2024-04-25T10:59:19.643222+0000","last_unstale":"2024-04-25T10:59:20.639078+0000","last_undegraded":"2024-04-25T10:59:20.639078+0000","last_fullsized":"2024-04-25T10:59:20.639078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:07.522883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.185571+0000","last_change":"2024-04-25T10:59:18.627083+0000","last_active":"2024-04-25T10:59:21.185571+0000","last_peered":"2024-04-25T10:59:21.185571+0000","last_clean":"2024-04-25T10:59:21.185571+0000","last_became_active":"2024-04-25T10:59:18.626450+0000","last_became_peered":"2024-04-25T10:59:18.626450+0000","last_unstale":"2024-04-25T10:59:21.185571+0000","last_undegraded":"2024-04-25T10:59:21.185571+0000","last_fullsized":"2024-04-25T10:59:21.185571+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:45.612221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228298+0000","last_change":"2024-04-25T10:59:19.653272+0000","last_active":"2024-04-25T10:59:21.228298+0000","last_peered":"2024-04-25T10:59:21.228298+0000","last_clean":"2024-04-25T10:59:21.228298+0000","last_became_active":"2024-04-25T10:59:19.652954+0000","last_became_peered":"2024-04-25T10:59:19.652954+0000","last_unstale":"2024-04-25T10:59:21.228298+0000","last_undegraded":"2024-04-25T10:59:21.228298+0000","last_fullsized":"2024-04-25T10:59:21.228298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:32.457404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638454+0000","last_change":"2024-04-25T10:59:18.630333+0000","last_active":"2024-04-25T10:59:20.638454+0000","last_peered":"2024-04-25T10:59:20.638454+0000","last_clean":"2024-04-25T10:59:20.638454+0000","last_became_active":"2024-04-25T10:59:18.629185+0000","last_became_peered":"2024-04-25T10:59:18.629185+0000","last_unstale":"2024-04-25T10:59:20.638454+0000","last_undegraded":"2024-04-25T10:59:20.638454+0000","last_fullsized":"2024-04-25T10:59:20.638454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:12.306399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228990+0000","last_change":"2024-04-25T10:59:19.654703+0000","last_active":"2024-04-25T10:59:21.228990+0000","last_peered":"2024-04-25T10:59:21.228990+0000","last_clean":"2024-04-25T10:59:21.228990+0000","last_became_active":"2024-04-25T10:59:19.654381+0000","last_became_peered":"2024-04-25T10:59:19.654381+0000","last_unstale":"2024-04-25T10:59:21.228990+0000","last_undegraded":"2024-04-25T10:59:21.228990+0000","last_fullsized":"2024-04-25T10:59:21.228990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:43.481266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.185751+0000","last_change":"2024-04-25T10:59:18.639591+0000","last_active":"2024-04-25T10:59:21.185751+0000","last_peered":"2024-04-25T10:59:21.185751+0000","last_clean":"2024-04-25T10:59:21.185751+0000","last_became_active":"2024-04-25T10:59:18.639276+0000","last_became_peered":"2024-04-25T10:59:18.639276+0000","last_unstale":"2024-04-25T10:59:21.185751+0000","last_undegraded":"2024-04-25T10:59:21.185751+0000","last_fullsized":"2024-04-25T10:59:21.185751+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:05.044315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.229162+0000","last_change":"2024-04-25T10:59:19.650496+0000","last_active":"2024-04-25T10:59:21.229162+0000","last_peered":"2024-04-25T10:59:21.229162+0000","last_clean":"2024-04-25T10:59:21.229162+0000","last_became_active":"2024-04-25T10:59:19.649725+0000","last_became_peered":"2024-04-25T10:59:19.649725+0000","last_unstale":"2024-04-25T10:59:21.229162+0000","last_undegraded":"2024-04-25T10:59:21.229162+0000","last_fullsized":"2024-04-25T10:59:21.229162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:53.860108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636578+0000","last_change":"2024-04-25T10:59:18.635647+0000","last_active":"2024-04-25T10:59:20.636578+0000","last_peered":"2024-04-25T10:59:20.636578+0000","last_clean":"2024-04-25T10:59:20.636578+0000","last_became_active":"2024-04-25T10:59:18.635264+0000","last_became_peered":"2024-04-25T10:59:18.635264+0000","last_unstale":"2024-04-25T10:59:20.636578+0000","last_undegraded":"2024-04-25T10:59:20.636578+0000","last_fullsized":"2024-04-25T10:59:20.636578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:45.612221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228718+0000","last_change":"2024-04-25T10:59:19.650869+0000","last_active":"2024-04-25T10:59:21.228718+0000","last_peered":"2024-04-25T10:59:21.228718+0000","last_clean":"2024-04-25T10:59:21.228718+0000","last_became_active":"2024-04-25T10:59:19.650541+0000","last_became_peered":"2024-04-25T10:59:19.650541+0000","last_unstale":"2024-04-25T10:59:21.228718+0000","last_undegraded":"2024-04-25T10:59:21.228718+0000","last_fullsized":"2024-04-25T10:59:21.228718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:14.189727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.185690+0000","last_change":"2024-04-25T10:59:18.637436+0000","last_active":"2024-04-25T10:59:21.185690+0000","last_peered":"2024-04-25T10:59:21.185690+0000","last_clean":"2024-04-25T10:59:21.185690+0000","last_became_active":"2024-04-25T10:59:18.637152+0000","last_became_peered":"2024-04-25T10:59:18.637152+0000","last_unstale":"2024-04-25T10:59:21.185690+0000","last_undegraded":"2024-04-25T10:59:21.185690+0000","last_fullsized":"2024-04-25T10:59:21.185690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:10.473815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228358+0000","last_change":"2024-04-25T10:59:19.655023+0000","last_active":"2024-04-25T10:59:21.228358+0000","last_peered":"2024-04-25T10:59:21.228358+0000","last_clean":"2024-04-25T10:59:21.228358+0000","last_became_active":"2024-04-25T10:59:19.654647+0000","last_became_peered":"2024-04-25T10:59:19.654647+0000","last_unstale":"2024-04-25T10:59:21.228358+0000","last_undegraded":"2024-04-25T10:59:21.228358+0000","last_fullsized":"2024-04-25T10:59:21.228358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:00.311029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":110,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227788+0000","last_change":"2024-04-25T10:59:07.839466+0000","last_active":"2024-04-25T10:59:21.227788+0000","last_peered":"2024-04-25T10:59:21.227788+0000","last_clean":"2024-04-25T10:59:21.227788+0000","last_became_active":"2024-04-25T10:59:07.836269+0000","last_became_peered":"2024-04-25T10:59:07.836269+0000","last_unstale":"2024-04-25T10:59:21.227788+0000","last_undegraded":"2024-04-25T10:59:21.227788+0000","last_fullsized":"2024-04-25T10:59:21.227788+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:06.549663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:06.549663+0000","last_clean_scrub_stamp":"2024-04-25T10:59:06.549663+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:21.575330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228265+0000","last_change":"2024-04-25T10:59:18.639570+0000","last_active":"2024-04-25T10:59:21.228265+0000","last_peered":"2024-04-25T10:59:21.228265+0000","last_clean":"2024-04-25T10:59:21.228265+0000","last_became_active":"2024-04-25T10:59:18.639231+0000","last_became_peered":"2024-04-25T10:59:18.639231+0000","last_unstale":"2024-04-25T10:59:21.228265+0000","last_undegraded":"2024-04-25T10:59:21.228265+0000","last_fullsized":"2024-04-25T10:59:21.228265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:28.594312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228305+0000","last_change":"2024-04-25T10:59:19.652991+0000","last_active":"2024-04-25T10:59:21.228305+0000","last_peered":"2024-04-25T10:59:21.228305+0000","last_clean":"2024-04-25T10:59:21.228305+0000","last_became_active":"2024-04-25T10:59:19.652680+0000","last_became_peered":"2024-04-25T10:59:19.652680+0000","last_unstale":"2024-04-25T10:59:21.228305+0000","last_undegraded":"2024-04-25T10:59:21.228305+0000","last_fullsized":"2024-04-25T10:59:21.228305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:38.297267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228139+0000","last_change":"2024-04-25T10:59:18.639571+0000","last_active":"2024-04-25T10:59:21.228139+0000","last_peered":"2024-04-25T10:59:21.228139+0000","last_clean":"2024-04-25T10:59:21.228139+0000","last_became_active":"2024-04-25T10:59:18.639065+0000","last_became_peered":"2024-04-25T10:59:18.639065+0000","last_unstale":"2024-04-25T10:59:21.228139+0000","last_undegraded":"2024-04-25T10:59:21.228139+0000","last_fullsized":"2024-04-25T10:59:21.228139+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:10.473815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227726+0000","last_change":"2024-04-25T10:59:19.644241+0000","last_active":"2024-04-25T10:59:21.227726+0000","last_peered":"2024-04-25T10:59:21.227726+0000","last_clean":"2024-04-25T10:59:21.227726+0000","last_became_active":"2024-04-25T10:59:19.643778+0000","last_became_peered":"2024-04-25T10:59:19.643778+0000","last_unstale":"2024-04-25T10:59:21.227726+0000","last_undegraded":"2024-04-25T10:59:21.227726+0000","last_fullsized":"2024-04-25T10:59:21.227726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:10.276726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.185642+0000","last_change":"2024-04-25T10:59:18.643811+0000","last_active":"2024-04-25T10:59:21.185642+0000","last_peered":"2024-04-25T10:59:21.185642+0000","last_clean":"2024-04-25T10:59:21.185642+0000","last_became_active":"2024-04-25T10:59:18.643454+0000","last_became_peered":"2024-04-25T10:59:18.643454+0000","last_unstale":"2024-04-25T10:59:21.185642+0000","last_undegraded":"2024-04-25T10:59:21.185642+0000","last_fullsized":"2024-04-25T10:59:21.185642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:09.933183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636785+0000","last_change":"2024-04-25T10:59:19.639259+0000","last_active":"2024-04-25T10:59:20.636785+0000","last_peered":"2024-04-25T10:59:20.636785+0000","last_clean":"2024-04-25T10:59:20.636785+0000","last_became_active":"2024-04-25T10:59:19.637680+0000","last_became_peered":"2024-04-25T10:59:19.637680+0000","last_unstale":"2024-04-25T10:59:20.636785+0000","last_undegraded":"2024-04-25T10:59:20.636785+0000","last_fullsized":"2024-04-25T10:59:20.636785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:33.050282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.229027+0000","last_change":"2024-04-25T10:59:18.643664+0000","last_active":"2024-04-25T10:59:21.229027+0000","last_peered":"2024-04-25T10:59:21.229027+0000","last_clean":"2024-04-25T10:59:21.229027+0000","last_became_active":"2024-04-25T10:59:18.643309+0000","last_became_peered":"2024-04-25T10:59:18.643309+0000","last_unstale":"2024-04-25T10:59:21.229027+0000","last_undegraded":"2024-04-25T10:59:21.229027+0000","last_fullsized":"2024-04-25T10:59:21.229027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:45.612221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.639084+0000","last_change":"2024-04-25T10:59:19.633507+0000","last_active":"2024-04-25T10:59:20.639084+0000","last_peered":"2024-04-25T10:59:20.639084+0000","last_clean":"2024-04-25T10:59:20.639084+0000","last_became_active":"2024-04-25T10:59:19.633182+0000","last_became_peered":"2024-04-25T10:59:19.633182+0000","last_unstale":"2024-04-25T10:59:20.639084+0000","last_undegraded":"2024-04-25T10:59:20.639084+0000","last_fullsized":"2024-04-25T10:59:20.639084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:11.481375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636672+0000","last_change":"2024-04-25T10:59:18.635772+0000","last_active":"2024-04-25T10:59:20.636672+0000","last_peered":"2024-04-25T10:59:20.636672+0000","last_clean":"2024-04-25T10:59:20.636672+0000","last_became_active":"2024-04-25T10:59:18.635486+0000","last_became_peered":"2024-04-25T10:59:18.635486+0000","last_unstale":"2024-04-25T10:59:20.636672+0000","last_undegraded":"2024-04-25T10:59:20.636672+0000","last_fullsized":"2024-04-25T10:59:20.636672+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:24.267022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228031+0000","last_change":"2024-04-25T10:59:19.648239+0000","last_active":"2024-04-25T10:59:21.228031+0000","last_peered":"2024-04-25T10:59:21.228031+0000","last_clean":"2024-04-25T10:59:21.228031+0000","last_became_active":"2024-04-25T10:59:19.646876+0000","last_became_peered":"2024-04-25T10:59:19.646876+0000","last_unstale":"2024-04-25T10:59:21.228031+0000","last_undegraded":"2024-04-25T10:59:21.228031+0000","last_fullsized":"2024-04-25T10:59:21.228031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:49.314956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228833+0000","last_change":"2024-04-25T10:59:18.643449+0000","last_active":"2024-04-25T10:59:21.228833+0000","last_peered":"2024-04-25T10:59:21.228833+0000","last_clean":"2024-04-25T10:59:21.228833+0000","last_became_active":"2024-04-25T10:59:18.643069+0000","last_became_peered":"2024-04-25T10:59:18.643069+0000","last_unstale":"2024-04-25T10:59:21.228833+0000","last_undegraded":"2024-04-25T10:59:21.228833+0000","last_fullsized":"2024-04-25T10:59:21.228833+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:57.176546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636600+0000","last_change":"2024-04-25T10:59:19.638854+0000","last_active":"2024-04-25T10:59:20.636600+0000","last_peered":"2024-04-25T10:59:20.636600+0000","last_clean":"2024-04-25T10:59:20.636600+0000","last_became_active":"2024-04-25T10:59:19.638273+0000","last_became_peered":"2024-04-25T10:59:19.638273+0000","last_unstale":"2024-04-25T10:59:20.636600+0000","last_undegraded":"2024-04-25T10:59:20.636600+0000","last_fullsized":"2024-04-25T10:59:20.636600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:15.132862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.428925+0000","last_change":"2024-04-25T10:59:18.635651+0000","last_active":"2024-04-25T10:59:21.428925+0000","last_peered":"2024-04-25T10:59:21.428925+0000","last_clean":"2024-04-25T10:59:21.428925+0000","last_became_active":"2024-04-25T10:59:18.635348+0000","last_became_peered":"2024-04-25T10:59:18.635348+0000","last_unstale":"2024-04-25T10:59:21.428925+0000","last_undegraded":"2024-04-25T10:59:21.428925+0000","last_fullsized":"2024-04-25T10:59:21.428925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:58.505038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228269+0000","last_change":"2024-04-25T10:59:19.640663+0000","last_active":"2024-04-25T10:59:21.228269+0000","last_peered":"2024-04-25T10:59:21.228269+0000","last_clean":"2024-04-25T10:59:21.228269+0000","last_became_active":"2024-04-25T10:59:19.640311+0000","last_became_peered":"2024-04-25T10:59:19.640311+0000","last_unstale":"2024-04-25T10:59:21.228269+0000","last_undegraded":"2024-04-25T10:59:21.228269+0000","last_fullsized":"2024-04-25T10:59:21.228269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:57.344787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638534+0000","last_change":"2024-04-25T10:59:18.635706+0000","last_active":"2024-04-25T10:59:20.638534+0000","last_peered":"2024-04-25T10:59:20.638534+0000","last_clean":"2024-04-25T10:59:20.638534+0000","last_became_active":"2024-04-25T10:59:18.635377+0000","last_became_peered":"2024-04-25T10:59:18.635377+0000","last_unstale":"2024-04-25T10:59:20.638534+0000","last_undegraded":"2024-04-25T10:59:20.638534+0000","last_fullsized":"2024-04-25T10:59:20.638534+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:24.267022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227935+0000","last_change":"2024-04-25T10:59:19.635255+0000","last_active":"2024-04-25T10:59:21.227935+0000","last_peered":"2024-04-25T10:59:21.227935+0000","last_clean":"2024-04-25T10:59:21.227935+0000","last_became_active":"2024-04-25T10:59:19.634829+0000","last_became_peered":"2024-04-25T10:59:19.634829+0000","last_unstale":"2024-04-25T10:59:21.227935+0000","last_undegraded":"2024-04-25T10:59:21.227935+0000","last_fullsized":"2024-04-25T10:59:21.227935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:52.815596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227702+0000","last_change":"2024-04-25T10:59:18.643806+0000","last_active":"2024-04-25T10:59:21.227702+0000","last_peered":"2024-04-25T10:59:21.227702+0000","last_clean":"2024-04-25T10:59:21.227702+0000","last_became_active":"2024-04-25T10:59:18.643374+0000","last_became_peered":"2024-04-25T10:59:18.643374+0000","last_unstale":"2024-04-25T10:59:21.227702+0000","last_undegraded":"2024-04-25T10:59:21.227702+0000","last_fullsized":"2024-04-25T10:59:21.227702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:45.612221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.637981+0000","last_change":"2024-04-25T10:59:19.637663+0000","last_active":"2024-04-25T10:59:20.637981+0000","last_peered":"2024-04-25T10:59:20.637981+0000","last_clean":"2024-04-25T10:59:20.637981+0000","last_became_active":"2024-04-25T10:59:19.637309+0000","last_became_peered":"2024-04-25T10:59:19.637309+0000","last_unstale":"2024-04-25T10:59:20.637981+0000","last_undegraded":"2024-04-25T10:59:20.637981+0000","last_fullsized":"2024-04-25T10:59:20.637981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:29.601873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636484+0000","last_change":"2024-04-25T10:59:18.636340+0000","last_active":"2024-04-25T10:59:20.636484+0000","last_peered":"2024-04-25T10:59:20.636484+0000","last_clean":"2024-04-25T10:59:20.636484+0000","last_became_active":"2024-04-25T10:59:18.636036+0000","last_became_peered":"2024-04-25T10:59:18.636036+0000","last_unstale":"2024-04-25T10:59:20.636484+0000","last_undegraded":"2024-04-25T10:59:20.636484+0000","last_fullsized":"2024-04-25T10:59:20.636484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:15.754476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227666+0000","last_change":"2024-04-25T10:59:19.632935+0000","last_active":"2024-04-25T10:59:21.227666+0000","last_peered":"2024-04-25T10:59:21.227666+0000","last_clean":"2024-04-25T10:59:21.227666+0000","last_became_active":"2024-04-25T10:59:19.632658+0000","last_became_peered":"2024-04-25T10:59:19.632658+0000","last_unstale":"2024-04-25T10:59:21.227666+0000","last_undegraded":"2024-04-25T10:59:21.227666+0000","last_fullsized":"2024-04-25T10:59:21.227666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:52.815596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636560+0000","last_change":"2024-04-25T10:59:18.639304+0000","last_active":"2024-04-25T10:59:20.636560+0000","last_peered":"2024-04-25T10:59:20.636560+0000","last_clean":"2024-04-25T10:59:20.636560+0000","last_became_active":"2024-04-25T10:59:18.638939+0000","last_became_peered":"2024-04-25T10:59:18.638939+0000","last_unstale":"2024-04-25T10:59:20.636560+0000","last_undegraded":"2024-04-25T10:59:20.636560+0000","last_fullsized":"2024-04-25T10:59:20.636560+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:28.594312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227866+0000","last_change":"2024-04-25T10:59:19.631658+0000","last_active":"2024-04-25T10:59:21.227866+0000","last_peered":"2024-04-25T10:59:21.227866+0000","last_clean":"2024-04-25T10:59:21.227866+0000","last_became_active":"2024-04-25T10:59:19.631363+0000","last_became_peered":"2024-04-25T10:59:19.631363+0000","last_unstale":"2024-04-25T10:59:21.227866+0000","last_undegraded":"2024-04-25T10:59:21.227866+0000","last_fullsized":"2024-04-25T10:59:21.227866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:33.050282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228855+0000","last_change":"2024-04-25T10:59:18.638514+0000","last_active":"2024-04-25T10:59:21.228855+0000","last_peered":"2024-04-25T10:59:21.228855+0000","last_clean":"2024-04-25T10:59:21.228855+0000","last_became_active":"2024-04-25T10:59:18.638113+0000","last_became_peered":"2024-04-25T10:59:18.638113+0000","last_unstale":"2024-04-25T10:59:21.228855+0000","last_undegraded":"2024-04-25T10:59:21.228855+0000","last_fullsized":"2024-04-25T10:59:21.228855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:09.933183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638752+0000","last_change":"2024-04-25T10:59:19.652586+0000","last_active":"2024-04-25T10:59:20.638752+0000","last_peered":"2024-04-25T10:59:20.638752+0000","last_clean":"2024-04-25T10:59:20.638752+0000","last_became_active":"2024-04-25T10:59:19.652246+0000","last_became_peered":"2024-04-25T10:59:19.652246+0000","last_unstale":"2024-04-25T10:59:20.638752+0000","last_undegraded":"2024-04-25T10:59:20.638752+0000","last_fullsized":"2024-04-25T10:59:20.638752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:52.815596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228042+0000","last_change":"2024-04-25T10:59:19.649474+0000","last_active":"2024-04-25T10:59:21.228042+0000","last_peered":"2024-04-25T10:59:21.228042+0000","last_clean":"2024-04-25T10:59:21.228042+0000","last_became_active":"2024-04-25T10:59:19.649134+0000","last_became_peered":"2024-04-25T10:59:19.649134+0000","last_unstale":"2024-04-25T10:59:21.228042+0000","last_undegraded":"2024-04-25T10:59:21.228042+0000","last_fullsized":"2024-04-25T10:59:21.228042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:38.297267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228094+0000","last_change":"2024-04-25T10:59:18.641120+0000","last_active":"2024-04-25T10:59:21.228094+0000","last_peered":"2024-04-25T10:59:21.228094+0000","last_clean":"2024-04-25T10:59:21.228094+0000","last_became_active":"2024-04-25T10:59:18.640791+0000","last_became_peered":"2024-04-25T10:59:18.640791+0000","last_unstale":"2024-04-25T10:59:21.228094+0000","last_undegraded":"2024-04-25T10:59:21.228094+0000","last_fullsized":"2024-04-25T10:59:21.228094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:43.470493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228726+0000","last_change":"2024-04-25T10:59:19.654395+0000","last_active":"2024-04-25T10:59:21.228726+0000","last_peered":"2024-04-25T10:59:21.228726+0000","last_clean":"2024-04-25T10:59:21.228726+0000","last_became_active":"2024-04-25T10:59:19.654107+0000","last_became_peered":"2024-04-25T10:59:19.654107+0000","last_unstale":"2024-04-25T10:59:21.228726+0000","last_undegraded":"2024-04-25T10:59:21.228726+0000","last_fullsized":"2024-04-25T10:59:21.228726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:33.154184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228501+0000","last_change":"2024-04-25T10:59:18.640843+0000","last_active":"2024-04-25T10:59:21.228501+0000","last_peered":"2024-04-25T10:59:21.228501+0000","last_clean":"2024-04-25T10:59:21.228501+0000","last_became_active":"2024-04-25T10:59:18.640496+0000","last_became_peered":"2024-04-25T10:59:18.640496+0000","last_unstale":"2024-04-25T10:59:21.228501+0000","last_undegraded":"2024-04-25T10:59:21.228501+0000","last_fullsized":"2024-04-25T10:59:21.228501+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:05.044315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227754+0000","last_change":"2024-04-25T10:59:19.652337+0000","last_active":"2024-04-25T10:59:21.227754+0000","last_peered":"2024-04-25T10:59:21.227754+0000","last_clean":"2024-04-25T10:59:21.227754+0000","last_became_active":"2024-04-25T10:59:19.652008+0000","last_became_peered":"2024-04-25T10:59:19.652008+0000","last_unstale":"2024-04-25T10:59:21.227754+0000","last_undegraded":"2024-04-25T10:59:21.227754+0000","last_fullsized":"2024-04-25T10:59:21.227754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:32:26.553467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228556+0000","last_change":"2024-04-25T10:59:18.643508+0000","last_active":"2024-04-25T10:59:21.228556+0000","last_peered":"2024-04-25T10:59:21.228556+0000","last_clean":"2024-04-25T10:59:21.228556+0000","last_became_active":"2024-04-25T10:59:18.643186+0000","last_became_peered":"2024-04-25T10:59:18.643186+0000","last_unstale":"2024-04-25T10:59:21.228556+0000","last_undegraded":"2024-04-25T10:59:21.228556+0000","last_fullsized":"2024-04-25T10:59:21.228556+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:35.341389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228500+0000","last_change":"2024-04-25T10:59:19.640778+0000","last_active":"2024-04-25T10:59:21.228500+0000","last_peered":"2024-04-25T10:59:21.228500+0000","last_clean":"2024-04-25T10:59:21.228500+0000","last_became_active":"2024-04-25T10:59:19.640479+0000","last_became_peered":"2024-04-25T10:59:19.640479+0000","last_unstale":"2024-04-25T10:59:21.228500+0000","last_undegraded":"2024-04-25T10:59:21.228500+0000","last_fullsized":"2024-04-25T10:59:21.228500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:52.334437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228246+0000","last_change":"2024-04-25T10:59:18.635569+0000","last_active":"2024-04-25T10:59:21.228246+0000","last_peered":"2024-04-25T10:59:21.228246+0000","last_clean":"2024-04-25T10:59:21.228246+0000","last_became_active":"2024-04-25T10:59:18.635303+0000","last_became_peered":"2024-04-25T10:59:18.635303+0000","last_unstale":"2024-04-25T10:59:21.228246+0000","last_undegraded":"2024-04-25T10:59:21.228246+0000","last_fullsized":"2024-04-25T10:59:21.228246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:46.279987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:59:25.441 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:59:25.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.584+0000 7f50fe451640 1 Processor -- start 2024-04-25T10:59:25.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.584+0000 7f50fe451640 1 -- start start 2024-04-25T10:59:25.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.585+0000 7f50fe451640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f50f80cfd30 0x7f50f80b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.585+0000 7f50fe451640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f50f80b3620 0x7f50f80b7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.585+0000 7f50fe451640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50f80cf210 con 0x7f50f80d0140 2024-04-25T10:59:25.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.585+0000 7f50fe451640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f50f80b1bc0 con 0x7f50f80cfd30 2024-04-25T10:59:25.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.585+0000 7f50fe451640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f50f80b3c40 con 0x7f50f80b3620 2024-04-25T10:59:25.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.585+0000 7f50fc9ee640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f50f80b3620 0x7f50f80b7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.585+0000 7f50f7fff640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f50f80cfd30 0x7f50f80b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.585+0000 7f50fc9ee640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f50f80b3620 0x7f50f80b7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.69:41688/0 (socket says 172.21.15.69:41688) 2024-04-25T10:59:25.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.585+0000 7f50fc9ee640 1 -- 172.21.15.69:0/1377341753 learned_addr learned my addr 172.21.15.69:0/1377341753 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:25.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.586+0000 7f50f7fff640 1 -- 172.21.15.69:0/1377341753 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f50f80b3620 msgr2=0x7f50f80b7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:25.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.586+0000 7f50f7fff640 1 --2- 172.21.15.69:0/1377341753 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f50f80b3620 0x7f50f80b7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.586+0000 7f50f7fff640 1 -- 172.21.15.69:0/1377341753 >> v1:172.21.15.146:6789/0 conn(0x7f50f80d0140 legacy=0x7f50f80d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.586+0000 7f50f7fff640 1 -- 172.21.15.69:0/1377341753 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f80b3ee0 con 0x7f50f80cfd30 2024-04-25T10:59:25.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.586+0000 7f50fc9ee640 1 --2- 172.21.15.69:0/1377341753 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f50f80b3620 0x7f50f80b7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:59:25.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.587+0000 7f50f7fff640 1 --2- 172.21.15.69:0/1377341753 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f50f80cfd30 0x7f50f80b77e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f50e80097a0 tx=0x7f50e802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=430cdd12f6d9ebde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:25.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.587+0000 7f50f6ffd640 1 -- 172.21.15.69:0/1377341753 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e8037070 con 0x7f50f80cfd30 2024-04-25T10:59:25.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.587+0000 7f50f6ffd640 1 -- 172.21.15.69:0/1377341753 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50e802ae50 con 0x7f50f80cfd30 2024-04-25T10:59:25.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.587+0000 7f50f6ffd640 1 -- 172.21.15.69:0/1377341753 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e8032660 con 0x7f50f80cfd30 2024-04-25T10:59:25.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.587+0000 7f50fe451640 1 -- 172.21.15.69:0/1377341753 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f50f80cfd30 msgr2=0x7f50f80b77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.587+0000 7f50fe451640 1 --2- 172.21.15.69:0/1377341753 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f50f80cfd30 0x7f50f80b77e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f50e80097a0 tx=0x7f50e802a100 comp rx=0 tx=0).stop 2024-04-25T10:59:25.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.587+0000 7f50fe451640 1 -- 172.21.15.69:0/1377341753 shutdown_connections 2024-04-25T10:59:25.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.587+0000 7f50fe451640 1 --2- 172.21.15.69:0/1377341753 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f50f80b3620 0x7f50f80b7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.587+0000 7f50fe451640 1 --2- 172.21.15.69:0/1377341753 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f50f80cfd30 0x7f50f80b77e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.588+0000 7f50fe451640 1 -- 172.21.15.69:0/1377341753 >> 172.21.15.69:0/1377341753 conn(0x7f50f809b7d0 msgr2=0x7f50f809bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:25.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.588+0000 7f50fe451640 1 -- 172.21.15.69:0/1377341753 shutdown_connections 2024-04-25T10:59:25.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.588+0000 7f50fe451640 1 -- 172.21.15.69:0/1377341753 wait complete. 2024-04-25T10:59:25.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.589+0000 7f50fe451640 1 Processor -- start 2024-04-25T10:59:25.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.589+0000 7f50fe451640 1 -- start start 2024-04-25T10:59:25.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.591+0000 7f50fe451640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f50f80b3620 0x7f50f8152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.591+0000 7f50fe451640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50f80cfd30 0x7f50f80cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.591+0000 7f50fe451640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f50f80d0140 0x7f50f80cb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.591+0000 7f50fe451640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f50f80b20f0 con 0x7f50f80cfd30 2024-04-25T10:59:25.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.591+0000 7f50fe451640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f50f80b1e10 con 0x7f50f80d0140 2024-04-25T10:59:25.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.591+0000 7f50fe451640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f50f81608f0 con 0x7f50f80b3620 2024-04-25T10:59:25.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.591+0000 7f50f77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50f80cfd30 0x7f50f80cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.591+0000 7f50f77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50f80cfd30 0x7f50f80cab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56246/0 (socket says 172.21.15.69:56246) 2024-04-25T10:59:25.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.591+0000 7f50f77fe640 1 -- 172.21.15.69:0/3825572268 learned_addr learned my addr 172.21.15.69:0/3825572268 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:25.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.591+0000 7f50f7fff640 1 --2- 172.21.15.69:0/3825572268 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f50f80b3620 0x7f50f8152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.591+0000 7f50fc9ee640 1 --2- 172.21.15.69:0/3825572268 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f50f80d0140 0x7f50f80cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.591+0000 7f50f77fe640 1 -- 172.21.15.69:0/3825572268 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f50f80b3620 msgr2=0x7f50f8152eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.591+0000 7f50f77fe640 1 --2- 172.21.15.69:0/3825572268 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f50f80b3620 0x7f50f8152eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.591+0000 7f50f77fe640 1 -- 172.21.15.69:0/3825572268 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f50f80d0140 msgr2=0x7f50f80cb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.591+0000 7f50f77fe640 1 --2- 172.21.15.69:0/3825572268 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f50f80d0140 0x7f50f80cb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.591+0000 7f50f77fe640 1 -- 172.21.15.69:0/3825572268 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f80c9800 con 0x7f50f80cfd30 2024-04-25T10:59:25.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.591+0000 7f50f77fe640 1 --2- 172.21.15.69:0/3825572268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50f80cfd30 0x7f50f80cab30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00c3c0 tx=0x7f50ec00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:25.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.591+0000 7f50f4ff9640 1 -- 172.21.15.69:0/3825572268 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec0052d0 con 0x7f50f80cfd30 2024-04-25T10:59:25.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.591+0000 7f50f4ff9640 1 -- 172.21.15.69:0/3825572268 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50ec005490 con 0x7f50f80cfd30 2024-04-25T10:59:25.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.591+0000 7f50fe451640 1 -- 172.21.15.69:0/3825572268 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f80c9520 con 0x7f50f80cfd30 2024-04-25T10:59:25.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.591+0000 7f50f4ff9640 1 -- 172.21.15.69:0/3825572268 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec007260 con 0x7f50f80cfd30 2024-04-25T10:59:25.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.592+0000 7f50fe451640 1 -- 172.21.15.69:0/3825572268 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f81ea6f0 con 0x7f50f80cfd30 2024-04-25T10:59:25.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.592+0000 7f50fe451640 1 -- 172.21.15.69:0/3825572268 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50f8050740 con 0x7f50f80cfd30 2024-04-25T10:59:25.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.593+0000 7f50f4ff9640 1 -- 172.21.15.69:0/3825572268 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f50ec007400 con 0x7f50f80cfd30 2024-04-25T10:59:25.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.594+0000 7f50f4ff9640 1 --2- 172.21.15.69:0/3825572268 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f50e40744b0 0x7f50e40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.594+0000 7f50f7fff640 1 --2- 172.21.15.69:0/3825572268 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f50e40744b0 0x7f50e40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.594+0000 7f50f4ff9640 1 -- 172.21.15.69:0/3825572268 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f50ec0946f0 con 0x7f50f80cfd30 2024-04-25T10:59:25.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.594+0000 7f50f7fff640 1 --2- 172.21.15.69:0/3825572268 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f50e40744b0 0x7f50e40769c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f50e8034000 tx=0x7f50e802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:25.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.598+0000 7f50f4ff9640 1 -- 172.21.15.69:0/3825572268 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f50ec05ee80 con 0x7f50f80cfd30 2024-04-25T10:59:25.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.770+0000 7f50fe451640 1 -- 172.21.15.69:0/3825572268 --> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f50f80c92b0 con 0x7f50e40744b0 2024-04-25T10:59:25.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.776+0000 7f50f4ff9640 1 -- 172.21.15.69:0/3825572268 <== mgr.4107 v2:172.21.15.69:6832/1107421267 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251825 (secure 0 0 0) 0x7f50f80c92b0 con 0x7f50e40744b0 2024-04-25T10:59:25.779 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:25.780 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-04-25T10:59:25.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.779+0000 7f50fe451640 1 -- 172.21.15.69:0/3825572268 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f50e40744b0 msgr2=0x7f50e40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.779+0000 7f50fe451640 1 --2- 172.21.15.69:0/3825572268 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f50e40744b0 0x7f50e40769c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f50e8034000 tx=0x7f50e802e000 comp rx=0 tx=0).stop 2024-04-25T10:59:25.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.779+0000 7f50fe451640 1 -- 172.21.15.69:0/3825572268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50f80cfd30 msgr2=0x7f50f80cab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.779+0000 7f50fe451640 1 --2- 172.21.15.69:0/3825572268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50f80cfd30 0x7f50f80cab30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00c3c0 tx=0x7f50ec00c880 comp rx=0 tx=0).stop 2024-04-25T10:59:25.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.779+0000 7f50fe451640 1 -- 172.21.15.69:0/3825572268 shutdown_connections 2024-04-25T10:59:25.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.779+0000 7f50fe451640 1 --2- 172.21.15.69:0/3825572268 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f50e40744b0 0x7f50e40769c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.779+0000 7f50fe451640 1 --2- 172.21.15.69:0/3825572268 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f50f80d0140 0x7f50f80cb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.779+0000 7f50fe451640 1 --2- 172.21.15.69:0/3825572268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50f80cfd30 0x7f50f80cab30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.779+0000 7f50fe451640 1 --2- 172.21.15.69:0/3825572268 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f50f80b3620 0x7f50f8152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.779+0000 7f50fe451640 1 -- 172.21.15.69:0/3825572268 >> 172.21.15.69:0/3825572268 conn(0x7f50f809b7d0 msgr2=0x7f50f8070470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:25.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.779+0000 7f50fe451640 1 -- 172.21.15.69:0/3825572268 shutdown_connections 2024-04-25T10:59:25.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.779+0000 7f50fe451640 1 -- 172.21.15.69:0/3825572268 wait complete. 2024-04-25T10:59:25.790 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-04-25T10:59:24.406977+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219012,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755708,"statfs":{"total":773094113280,"available":772869844992,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001259"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228976+0000","last_change":"2024-04-25T10:59:19.640957+0000","last_active":"2024-04-25T10:59:21.228976+0000","last_peered":"2024-04-25T10:59:21.228976+0000","last_clean":"2024-04-25T10:59:21.228976+0000","last_became_active":"2024-04-25T10:59:19.640610+0000","last_became_peered":"2024-04-25T10:59:19.640610+0000","last_unstale":"2024-04-25T10:59:21.228976+0000","last_undegraded":"2024-04-25T10:59:21.228976+0000","last_fullsized":"2024-04-25T10:59:21.228976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:42.867755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636604+0000","last_change":"2024-04-25T10:59:19.644473+0000","last_active":"2024-04-25T10:59:20.636604+0000","last_peered":"2024-04-25T10:59:20.636604+0000","last_clean":"2024-04-25T10:59:20.636604+0000","last_became_active":"2024-04-25T10:59:19.644151+0000","last_became_peered":"2024-04-25T10:59:19.644151+0000","last_unstale":"2024-04-25T10:59:20.636604+0000","last_undegraded":"2024-04-25T10:59:20.636604+0000","last_fullsized":"2024-04-25T10:59:20.636604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:07.522883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227761+0000","last_change":"2024-04-25T10:59:19.630984+0000","last_active":"2024-04-25T10:59:21.227761+0000","last_peered":"2024-04-25T10:59:21.227761+0000","last_clean":"2024-04-25T10:59:21.227761+0000","last_became_active":"2024-04-25T10:59:19.630699+0000","last_became_peered":"2024-04-25T10:59:19.630699+0000","last_unstale":"2024-04-25T10:59:21.227761+0000","last_undegraded":"2024-04-25T10:59:21.227761+0000","last_fullsized":"2024-04-25T10:59:21.227761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:59.213210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228437+0000","last_change":"2024-04-25T10:59:19.649708+0000","last_active":"2024-04-25T10:59:21.228437+0000","last_peered":"2024-04-25T10:59:21.228437+0000","last_clean":"2024-04-25T10:59:21.228437+0000","last_became_active":"2024-04-25T10:59:19.649262+0000","last_became_peered":"2024-04-25T10:59:19.649262+0000","last_unstale":"2024-04-25T10:59:21.228437+0000","last_undegraded":"2024-04-25T10:59:21.228437+0000","last_fullsized":"2024-04-25T10:59:21.228437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:52.334437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228519+0000","last_change":"2024-04-25T10:59:19.642490+0000","last_active":"2024-04-25T10:59:21.228519+0000","last_peered":"2024-04-25T10:59:21.228519+0000","last_clean":"2024-04-25T10:59:21.228519+0000","last_became_active":"2024-04-25T10:59:19.642122+0000","last_became_peered":"2024-04-25T10:59:19.642122+0000","last_unstale":"2024-04-25T10:59:21.228519+0000","last_undegraded":"2024-04-25T10:59:21.228519+0000","last_fullsized":"2024-04-25T10:59:21.228519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:14.189727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228892+0000","last_change":"2024-04-25T10:59:19.651061+0000","last_active":"2024-04-25T10:59:21.228892+0000","last_peered":"2024-04-25T10:59:21.228892+0000","last_clean":"2024-04-25T10:59:21.228892+0000","last_became_active":"2024-04-25T10:59:19.650710+0000","last_became_peered":"2024-04-25T10:59:19.650710+0000","last_unstale":"2024-04-25T10:59:21.228892+0000","last_undegraded":"2024-04-25T10:59:21.228892+0000","last_fullsized":"2024-04-25T10:59:21.228892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:45.328749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227817+0000","last_change":"2024-04-25T10:59:19.646862+0000","last_active":"2024-04-25T10:59:21.227817+0000","last_peered":"2024-04-25T10:59:21.227817+0000","last_clean":"2024-04-25T10:59:21.227817+0000","last_became_active":"2024-04-25T10:59:19.646553+0000","last_became_peered":"2024-04-25T10:59:19.646553+0000","last_unstale":"2024-04-25T10:59:21.227817+0000","last_undegraded":"2024-04-25T10:59:21.227817+0000","last_fullsized":"2024-04-25T10:59:21.227817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:52.334437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227508+0000","last_change":"2024-04-25T10:59:19.648086+0000","last_active":"2024-04-25T10:59:21.227508+0000","last_peered":"2024-04-25T10:59:21.227508+0000","last_clean":"2024-04-25T10:59:21.227508+0000","last_became_active":"2024-04-25T10:59:19.647702+0000","last_became_peered":"2024-04-25T10:59:19.647702+0000","last_unstale":"2024-04-25T10:59:21.227508+0000","last_undegraded":"2024-04-25T10:59:21.227508+0000","last_fullsized":"2024-04-25T10:59:21.227508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:59.213210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228983+0000","last_change":"2024-04-25T10:59:19.655202+0000","last_active":"2024-04-25T10:59:21.228983+0000","last_peered":"2024-04-25T10:59:21.228983+0000","last_clean":"2024-04-25T10:59:21.228983+0000","last_became_active":"2024-04-25T10:59:19.654871+0000","last_became_peered":"2024-04-25T10:59:19.654871+0000","last_unstale":"2024-04-25T10:59:21.228983+0000","last_undegraded":"2024-04-25T10:59:21.228983+0000","last_fullsized":"2024-04-25T10:59:21.228983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:50.893634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638094+0000","last_change":"2024-04-25T10:59:19.651928+0000","last_active":"2024-04-25T10:59:20.638094+0000","last_peered":"2024-04-25T10:59:20.638094+0000","last_clean":"2024-04-25T10:59:20.638094+0000","last_became_active":"2024-04-25T10:59:19.651618+0000","last_became_peered":"2024-04-25T10:59:19.651618+0000","last_unstale":"2024-04-25T10:59:20.638094+0000","last_undegraded":"2024-04-25T10:59:20.638094+0000","last_fullsized":"2024-04-25T10:59:20.638094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:32:26.553467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638187+0000","last_change":"2024-04-25T10:59:19.652810+0000","last_active":"2024-04-25T10:59:20.638187+0000","last_peered":"2024-04-25T10:59:20.638187+0000","last_clean":"2024-04-25T10:59:20.638187+0000","last_became_active":"2024-04-25T10:59:19.652118+0000","last_became_peered":"2024-04-25T10:59:19.652118+0000","last_unstale":"2024-04-25T10:59:20.638187+0000","last_undegraded":"2024-04-25T10:59:20.638187+0000","last_fullsized":"2024-04-25T10:59:20.638187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:59.213210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638421+0000","last_change":"2024-04-25T10:59:19.645548+0000","last_active":"2024-04-25T10:59:20.638421+0000","last_peered":"2024-04-25T10:59:20.638421+0000","last_clean":"2024-04-25T10:59:20.638421+0000","last_became_active":"2024-04-25T10:59:19.645231+0000","last_became_peered":"2024-04-25T10:59:19.645231+0000","last_unstale":"2024-04-25T10:59:20.638421+0000","last_undegraded":"2024-04-25T10:59:20.638421+0000","last_fullsized":"2024-04-25T10:59:20.638421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:59.512598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227723+0000","last_change":"2024-04-25T10:59:19.640736+0000","last_active":"2024-04-25T10:59:21.227723+0000","last_peered":"2024-04-25T10:59:21.227723+0000","last_clean":"2024-04-25T10:59:21.227723+0000","last_became_active":"2024-04-25T10:59:19.640448+0000","last_became_peered":"2024-04-25T10:59:19.640448+0000","last_unstale":"2024-04-25T10:59:21.227723+0000","last_undegraded":"2024-04-25T10:59:21.227723+0000","last_fullsized":"2024-04-25T10:59:21.227723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:15.132862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228215+0000","last_change":"2024-04-25T10:59:19.634929+0000","last_active":"2024-04-25T10:59:21.228215+0000","last_peered":"2024-04-25T10:59:21.228215+0000","last_clean":"2024-04-25T10:59:21.228215+0000","last_became_active":"2024-04-25T10:59:19.634640+0000","last_became_peered":"2024-04-25T10:59:19.634640+0000","last_unstale":"2024-04-25T10:59:21.228215+0000","last_undegraded":"2024-04-25T10:59:21.228215+0000","last_fullsized":"2024-04-25T10:59:21.228215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:49.314956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636375+0000","last_change":"2024-04-25T10:59:19.648597+0000","last_active":"2024-04-25T10:59:20.636375+0000","last_peered":"2024-04-25T10:59:20.636375+0000","last_clean":"2024-04-25T10:59:20.636375+0000","last_became_active":"2024-04-25T10:59:19.648280+0000","last_became_peered":"2024-04-25T10:59:19.648280+0000","last_unstale":"2024-04-25T10:59:20.636375+0000","last_undegraded":"2024-04-25T10:59:20.636375+0000","last_fullsized":"2024-04-25T10:59:20.636375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:58.824966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228034+0000","last_change":"2024-04-25T10:59:19.647183+0000","last_active":"2024-04-25T10:59:21.228034+0000","last_peered":"2024-04-25T10:59:21.228034+0000","last_clean":"2024-04-25T10:59:21.228034+0000","last_became_active":"2024-04-25T10:59:19.646563+0000","last_became_peered":"2024-04-25T10:59:19.646563+0000","last_unstale":"2024-04-25T10:59:21.228034+0000","last_undegraded":"2024-04-25T10:59:21.228034+0000","last_fullsized":"2024-04-25T10:59:21.228034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:57.344787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228532+0000","last_change":"2024-04-25T10:59:19.649207+0000","last_active":"2024-04-25T10:59:21.228532+0000","last_peered":"2024-04-25T10:59:21.228532+0000","last_clean":"2024-04-25T10:59:21.228532+0000","last_became_active":"2024-04-25T10:59:19.647280+0000","last_became_peered":"2024-04-25T10:59:19.647280+0000","last_unstale":"2024-04-25T10:59:21.228532+0000","last_undegraded":"2024-04-25T10:59:21.228532+0000","last_fullsized":"2024-04-25T10:59:21.228532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:32.457404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.185793+0000","last_change":"2024-04-25T10:59:19.640810+0000","last_active":"2024-04-25T10:59:21.185793+0000","last_peered":"2024-04-25T10:59:21.185793+0000","last_clean":"2024-04-25T10:59:21.185793+0000","last_became_active":"2024-04-25T10:59:19.640524+0000","last_became_peered":"2024-04-25T10:59:19.640524+0000","last_unstale":"2024-04-25T10:59:21.185793+0000","last_undegraded":"2024-04-25T10:59:21.185793+0000","last_fullsized":"2024-04-25T10:59:21.185793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:58.824966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228703+0000","last_change":"2024-04-25T10:59:19.641698+0000","last_active":"2024-04-25T10:59:21.228703+0000","last_peered":"2024-04-25T10:59:21.228703+0000","last_clean":"2024-04-25T10:59:21.228703+0000","last_became_active":"2024-04-25T10:59:19.640996+0000","last_became_peered":"2024-04-25T10:59:19.640996+0000","last_unstale":"2024-04-25T10:59:21.228703+0000","last_undegraded":"2024-04-25T10:59:21.228703+0000","last_fullsized":"2024-04-25T10:59:21.228703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:33.154184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.639226+0000","last_change":"2024-04-25T10:59:19.654176+0000","last_active":"2024-04-25T10:59:20.639226+0000","last_peered":"2024-04-25T10:59:20.639226+0000","last_clean":"2024-04-25T10:59:20.639226+0000","last_became_active":"2024-04-25T10:59:19.653175+0000","last_became_peered":"2024-04-25T10:59:19.653175+0000","last_unstale":"2024-04-25T10:59:20.639226+0000","last_undegraded":"2024-04-25T10:59:20.639226+0000","last_fullsized":"2024-04-25T10:59:20.639226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:33.050282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227935+0000","last_change":"2024-04-25T10:59:19.642959+0000","last_active":"2024-04-25T10:59:21.227935+0000","last_peered":"2024-04-25T10:59:21.227935+0000","last_clean":"2024-04-25T10:59:21.227935+0000","last_became_active":"2024-04-25T10:59:19.642671+0000","last_became_peered":"2024-04-25T10:59:19.642671+0000","last_unstale":"2024-04-25T10:59:21.227935+0000","last_undegraded":"2024-04-25T10:59:21.227935+0000","last_fullsized":"2024-04-25T10:59:21.227935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:15.132862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228503+0000","last_change":"2024-04-25T10:59:19.658039+0000","last_active":"2024-04-25T10:59:21.228503+0000","last_peered":"2024-04-25T10:59:21.228503+0000","last_clean":"2024-04-25T10:59:21.228503+0000","last_became_active":"2024-04-25T10:59:19.657194+0000","last_became_peered":"2024-04-25T10:59:19.657194+0000","last_unstale":"2024-04-25T10:59:21.228503+0000","last_undegraded":"2024-04-25T10:59:21.228503+0000","last_fullsized":"2024-04-25T10:59:21.228503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:19.485287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227782+0000","last_change":"2024-04-25T10:59:19.645451+0000","last_active":"2024-04-25T10:59:21.227782+0000","last_peered":"2024-04-25T10:59:21.227782+0000","last_clean":"2024-04-25T10:59:21.227782+0000","last_became_active":"2024-04-25T10:59:19.645121+0000","last_became_peered":"2024-04-25T10:59:19.645121+0000","last_unstale":"2024-04-25T10:59:21.227782+0000","last_undegraded":"2024-04-25T10:59:21.227782+0000","last_fullsized":"2024-04-25T10:59:21.227782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:07.522883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228328+0000","last_change":"2024-04-25T10:59:19.646916+0000","last_active":"2024-04-25T10:59:21.228328+0000","last_peered":"2024-04-25T10:59:21.228328+0000","last_clean":"2024-04-25T10:59:21.228328+0000","last_became_active":"2024-04-25T10:59:19.646228+0000","last_became_peered":"2024-04-25T10:59:19.646228+0000","last_unstale":"2024-04-25T10:59:21.228328+0000","last_undegraded":"2024-04-25T10:59:21.228328+0000","last_fullsized":"2024-04-25T10:59:21.228328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:17.431757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228328+0000","last_change":"2024-04-25T10:59:19.644795+0000","last_active":"2024-04-25T10:59:21.228328+0000","last_peered":"2024-04-25T10:59:21.228328+0000","last_clean":"2024-04-25T10:59:21.228328+0000","last_became_active":"2024-04-25T10:59:19.644503+0000","last_became_peered":"2024-04-25T10:59:19.644503+0000","last_unstale":"2024-04-25T10:59:21.228328+0000","last_undegraded":"2024-04-25T10:59:21.228328+0000","last_fullsized":"2024-04-25T10:59:21.228328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:17.431757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638291+0000","last_change":"2024-04-25T10:59:19.650630+0000","last_active":"2024-04-25T10:59:20.638291+0000","last_peered":"2024-04-25T10:59:20.638291+0000","last_clean":"2024-04-25T10:59:20.638291+0000","last_became_active":"2024-04-25T10:59:19.650312+0000","last_became_peered":"2024-04-25T10:59:19.650312+0000","last_unstale":"2024-04-25T10:59:20.638291+0000","last_undegraded":"2024-04-25T10:59:20.638291+0000","last_fullsized":"2024-04-25T10:59:20.638291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:58.824966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636446+0000","last_change":"2024-04-25T10:59:19.641641+0000","last_active":"2024-04-25T10:59:20.636446+0000","last_peered":"2024-04-25T10:59:20.636446+0000","last_clean":"2024-04-25T10:59:20.636446+0000","last_became_active":"2024-04-25T10:59:19.641339+0000","last_became_peered":"2024-04-25T10:59:19.641339+0000","last_unstale":"2024-04-25T10:59:20.636446+0000","last_undegraded":"2024-04-25T10:59:20.636446+0000","last_fullsized":"2024-04-25T10:59:20.636446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:52.815596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228173+0000","last_change":"2024-04-25T10:59:19.642824+0000","last_active":"2024-04-25T10:59:21.228173+0000","last_peered":"2024-04-25T10:59:21.228173+0000","last_clean":"2024-04-25T10:59:21.228173+0000","last_became_active":"2024-04-25T10:59:19.641152+0000","last_became_peered":"2024-04-25T10:59:19.641152+0000","last_unstale":"2024-04-25T10:59:21.228173+0000","last_undegraded":"2024-04-25T10:59:21.228173+0000","last_fullsized":"2024-04-25T10:59:21.228173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:49.314956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228117+0000","last_change":"2024-04-25T10:59:19.629558+0000","last_active":"2024-04-25T10:59:21.228117+0000","last_peered":"2024-04-25T10:59:21.228117+0000","last_clean":"2024-04-25T10:59:21.228117+0000","last_became_active":"2024-04-25T10:59:19.628864+0000","last_became_peered":"2024-04-25T10:59:19.628864+0000","last_unstale":"2024-04-25T10:59:21.228117+0000","last_undegraded":"2024-04-25T10:59:21.228117+0000","last_fullsized":"2024-04-25T10:59:21.228117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:07.522883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228910+0000","last_change":"2024-04-25T10:59:19.648576+0000","last_active":"2024-04-25T10:59:21.228910+0000","last_peered":"2024-04-25T10:59:21.228910+0000","last_clean":"2024-04-25T10:59:21.228910+0000","last_became_active":"2024-04-25T10:59:19.647435+0000","last_became_peered":"2024-04-25T10:59:19.647435+0000","last_unstale":"2024-04-25T10:59:21.228910+0000","last_undegraded":"2024-04-25T10:59:21.228910+0000","last_fullsized":"2024-04-25T10:59:21.228910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:42.867755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.185603+0000","last_change":"2024-04-25T10:59:19.649031+0000","last_active":"2024-04-25T10:59:21.185603+0000","last_peered":"2024-04-25T10:59:21.185603+0000","last_clean":"2024-04-25T10:59:21.185603+0000","last_became_active":"2024-04-25T10:59:19.648698+0000","last_became_peered":"2024-04-25T10:59:19.648698+0000","last_unstale":"2024-04-25T10:59:21.185603+0000","last_undegraded":"2024-04-25T10:59:21.185603+0000","last_fullsized":"2024-04-25T10:59:21.185603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:59.213210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638942+0000","last_change":"2024-04-25T10:59:19.643547+0000","last_active":"2024-04-25T10:59:20.638942+0000","last_peered":"2024-04-25T10:59:20.638942+0000","last_clean":"2024-04-25T10:59:20.638942+0000","last_became_active":"2024-04-25T10:59:19.642802+0000","last_became_peered":"2024-04-25T10:59:19.642802+0000","last_unstale":"2024-04-25T10:59:20.638942+0000","last_undegraded":"2024-04-25T10:59:20.638942+0000","last_fullsized":"2024-04-25T10:59:20.638942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:15.132862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638161+0000","last_change":"2024-04-25T10:59:19.638090+0000","last_active":"2024-04-25T10:59:20.638161+0000","last_peered":"2024-04-25T10:59:20.638161+0000","last_clean":"2024-04-25T10:59:20.638161+0000","last_became_active":"2024-04-25T10:59:19.637745+0000","last_became_peered":"2024-04-25T10:59:19.637745+0000","last_unstale":"2024-04-25T10:59:20.638161+0000","last_undegraded":"2024-04-25T10:59:20.638161+0000","last_fullsized":"2024-04-25T10:59:20.638161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:25.274583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227963+0000","last_change":"2024-04-25T10:59:18.637294+0000","last_active":"2024-04-25T10:59:21.227963+0000","last_peered":"2024-04-25T10:59:21.227963+0000","last_clean":"2024-04-25T10:59:21.227963+0000","last_became_active":"2024-04-25T10:59:18.636963+0000","last_became_peered":"2024-04-25T10:59:18.636963+0000","last_unstale":"2024-04-25T10:59:21.227963+0000","last_undegraded":"2024-04-25T10:59:21.227963+0000","last_fullsized":"2024-04-25T10:59:21.227963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:18.298759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228183+0000","last_change":"2024-04-25T10:59:19.638014+0000","last_active":"2024-04-25T10:59:21.228183+0000","last_peered":"2024-04-25T10:59:21.228183+0000","last_clean":"2024-04-25T10:59:21.228183+0000","last_became_active":"2024-04-25T10:59:19.637471+0000","last_became_peered":"2024-04-25T10:59:19.637471+0000","last_unstale":"2024-04-25T10:59:21.228183+0000","last_undegraded":"2024-04-25T10:59:21.228183+0000","last_fullsized":"2024-04-25T10:59:21.228183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:33.050282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638880+0000","last_change":"2024-04-25T10:59:18.634982+0000","last_active":"2024-04-25T10:59:20.638880+0000","last_peered":"2024-04-25T10:59:20.638880+0000","last_clean":"2024-04-25T10:59:20.638880+0000","last_became_active":"2024-04-25T10:59:18.634658+0000","last_became_peered":"2024-04-25T10:59:18.634658+0000","last_unstale":"2024-04-25T10:59:20.638880+0000","last_undegraded":"2024-04-25T10:59:20.638880+0000","last_fullsized":"2024-04-25T10:59:20.638880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:35.341389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227940+0000","last_change":"2024-04-25T10:59:18.632617+0000","last_active":"2024-04-25T10:59:21.227940+0000","last_peered":"2024-04-25T10:59:21.227940+0000","last_clean":"2024-04-25T10:59:21.227940+0000","last_became_active":"2024-04-25T10:59:18.632294+0000","last_became_peered":"2024-04-25T10:59:18.632294+0000","last_unstale":"2024-04-25T10:59:21.227940+0000","last_undegraded":"2024-04-25T10:59:21.227940+0000","last_fullsized":"2024-04-25T10:59:21.227940+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:46.279987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227575+0000","last_change":"2024-04-25T10:59:19.632333+0000","last_active":"2024-04-25T10:59:21.227575+0000","last_peered":"2024-04-25T10:59:21.227575+0000","last_clean":"2024-04-25T10:59:21.227575+0000","last_became_active":"2024-04-25T10:59:19.632042+0000","last_became_peered":"2024-04-25T10:59:19.632042+0000","last_unstale":"2024-04-25T10:59:21.227575+0000","last_undegraded":"2024-04-25T10:59:21.227575+0000","last_fullsized":"2024-04-25T10:59:21.227575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:58.824966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228184+0000","last_change":"2024-04-25T10:59:18.638103+0000","last_active":"2024-04-25T10:59:21.228184+0000","last_peered":"2024-04-25T10:59:21.228184+0000","last_clean":"2024-04-25T10:59:21.228184+0000","last_became_active":"2024-04-25T10:59:18.636931+0000","last_became_peered":"2024-04-25T10:59:18.636931+0000","last_unstale":"2024-04-25T10:59:21.228184+0000","last_undegraded":"2024-04-25T10:59:21.228184+0000","last_fullsized":"2024-04-25T10:59:21.228184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:15.754476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636374+0000","last_change":"2024-04-25T10:59:19.633809+0000","last_active":"2024-04-25T10:59:20.636374+0000","last_peered":"2024-04-25T10:59:20.636374+0000","last_clean":"2024-04-25T10:59:20.636374+0000","last_became_active":"2024-04-25T10:59:19.632787+0000","last_became_peered":"2024-04-25T10:59:19.632787+0000","last_unstale":"2024-04-25T10:59:20.636374+0000","last_undegraded":"2024-04-25T10:59:20.636374+0000","last_fullsized":"2024-04-25T10:59:20.636374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:59.213210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638419+0000","last_change":"2024-04-25T10:59:18.641813+0000","last_active":"2024-04-25T10:59:20.638419+0000","last_peered":"2024-04-25T10:59:20.638419+0000","last_clean":"2024-04-25T10:59:20.638419+0000","last_became_active":"2024-04-25T10:59:18.641127+0000","last_became_peered":"2024-04-25T10:59:18.641127+0000","last_unstale":"2024-04-25T10:59:20.638419+0000","last_undegraded":"2024-04-25T10:59:20.638419+0000","last_fullsized":"2024-04-25T10:59:20.638419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:28.594312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228699+0000","last_change":"2024-04-25T10:59:19.645823+0000","last_active":"2024-04-25T10:59:21.228699+0000","last_peered":"2024-04-25T10:59:21.228699+0000","last_clean":"2024-04-25T10:59:21.228699+0000","last_became_active":"2024-04-25T10:59:19.645500+0000","last_became_peered":"2024-04-25T10:59:19.645500+0000","last_unstale":"2024-04-25T10:59:21.228699+0000","last_undegraded":"2024-04-25T10:59:21.228699+0000","last_fullsized":"2024-04-25T10:59:21.228699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:00.311029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228172+0000","last_change":"2024-04-25T10:59:18.641002+0000","last_active":"2024-04-25T10:59:21.228172+0000","last_peered":"2024-04-25T10:59:21.228172+0000","last_clean":"2024-04-25T10:59:21.228172+0000","last_became_active":"2024-04-25T10:59:18.640589+0000","last_became_peered":"2024-04-25T10:59:18.640589+0000","last_unstale":"2024-04-25T10:59:21.228172+0000","last_undegraded":"2024-04-25T10:59:21.228172+0000","last_fullsized":"2024-04-25T10:59:21.228172+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:24.267022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227906+0000","last_change":"2024-04-25T10:59:19.652033+0000","last_active":"2024-04-25T10:59:21.227906+0000","last_peered":"2024-04-25T10:59:21.227906+0000","last_clean":"2024-04-25T10:59:21.227906+0000","last_became_active":"2024-04-25T10:59:19.651694+0000","last_became_peered":"2024-04-25T10:59:19.651694+0000","last_unstale":"2024-04-25T10:59:21.227906+0000","last_undegraded":"2024-04-25T10:59:21.227906+0000","last_fullsized":"2024-04-25T10:59:21.227906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:58.824966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636485+0000","last_change":"2024-04-25T10:59:18.638096+0000","last_active":"2024-04-25T10:59:20.636485+0000","last_peered":"2024-04-25T10:59:20.636485+0000","last_clean":"2024-04-25T10:59:20.636485+0000","last_became_active":"2024-04-25T10:59:18.637808+0000","last_became_peered":"2024-04-25T10:59:18.637808+0000","last_unstale":"2024-04-25T10:59:20.636485+0000","last_undegraded":"2024-04-25T10:59:20.636485+0000","last_fullsized":"2024-04-25T10:59:20.636485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:10.473815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638613+0000","last_change":"2024-04-25T10:59:19.638709+0000","last_active":"2024-04-25T10:59:20.638613+0000","last_peered":"2024-04-25T10:59:20.638613+0000","last_clean":"2024-04-25T10:59:20.638613+0000","last_became_active":"2024-04-25T10:59:19.638358+0000","last_became_peered":"2024-04-25T10:59:19.638358+0000","last_unstale":"2024-04-25T10:59:20.638613+0000","last_undegraded":"2024-04-25T10:59:20.638613+0000","last_fullsized":"2024-04-25T10:59:20.638613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:46.619782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.639204+0000","last_change":"2024-04-25T10:59:18.639824+0000","last_active":"2024-04-25T10:59:20.639204+0000","last_peered":"2024-04-25T10:59:20.639204+0000","last_clean":"2024-04-25T10:59:20.639204+0000","last_became_active":"2024-04-25T10:59:18.639316+0000","last_became_peered":"2024-04-25T10:59:18.639316+0000","last_unstale":"2024-04-25T10:59:20.639204+0000","last_undegraded":"2024-04-25T10:59:20.639204+0000","last_fullsized":"2024-04-25T10:59:20.639204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:15.754476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228064+0000","last_change":"2024-04-25T10:59:19.641380+0000","last_active":"2024-04-25T10:59:21.228064+0000","last_peered":"2024-04-25T10:59:21.228064+0000","last_clean":"2024-04-25T10:59:21.228064+0000","last_became_active":"2024-04-25T10:59:19.640580+0000","last_became_peered":"2024-04-25T10:59:19.640580+0000","last_unstale":"2024-04-25T10:59:21.228064+0000","last_undegraded":"2024-04-25T10:59:21.228064+0000","last_fullsized":"2024-04-25T10:59:21.228064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:19.485287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228226+0000","last_change":"2024-04-25T10:59:18.634324+0000","last_active":"2024-04-25T10:59:21.228226+0000","last_peered":"2024-04-25T10:59:21.228226+0000","last_clean":"2024-04-25T10:59:21.228226+0000","last_became_active":"2024-04-25T10:59:18.634049+0000","last_became_peered":"2024-04-25T10:59:18.634049+0000","last_unstale":"2024-04-25T10:59:21.228226+0000","last_undegraded":"2024-04-25T10:59:21.228226+0000","last_fullsized":"2024-04-25T10:59:21.228226+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:02.619679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228376+0000","last_change":"2024-04-25T10:59:19.643539+0000","last_active":"2024-04-25T10:59:21.228376+0000","last_peered":"2024-04-25T10:59:21.228376+0000","last_clean":"2024-04-25T10:59:21.228376+0000","last_became_active":"2024-04-25T10:59:19.643182+0000","last_became_peered":"2024-04-25T10:59:19.643182+0000","last_unstale":"2024-04-25T10:59:21.228376+0000","last_undegraded":"2024-04-25T10:59:21.228376+0000","last_fullsized":"2024-04-25T10:59:21.228376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:57.344787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.229036+0000","last_change":"2024-04-25T10:59:18.643936+0000","last_active":"2024-04-25T10:59:21.229036+0000","last_peered":"2024-04-25T10:59:21.229036+0000","last_clean":"2024-04-25T10:59:21.229036+0000","last_became_active":"2024-04-25T10:59:18.643576+0000","last_became_peered":"2024-04-25T10:59:18.643576+0000","last_unstale":"2024-04-25T10:59:21.229036+0000","last_undegraded":"2024-04-25T10:59:21.229036+0000","last_fullsized":"2024-04-25T10:59:21.229036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:58.505038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228718+0000","last_change":"2024-04-25T10:59:19.652528+0000","last_active":"2024-04-25T10:59:21.228718+0000","last_peered":"2024-04-25T10:59:21.228718+0000","last_clean":"2024-04-25T10:59:21.228718+0000","last_became_active":"2024-04-25T10:59:19.651277+0000","last_became_peered":"2024-04-25T10:59:19.651277+0000","last_unstale":"2024-04-25T10:59:21.228718+0000","last_undegraded":"2024-04-25T10:59:21.228718+0000","last_fullsized":"2024-04-25T10:59:21.228718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:45.328749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.637715+0000","last_change":"2024-04-25T10:59:18.630832+0000","last_active":"2024-04-25T10:59:20.637715+0000","last_peered":"2024-04-25T10:59:20.637715+0000","last_clean":"2024-04-25T10:59:20.637715+0000","last_became_active":"2024-04-25T10:59:18.630462+0000","last_became_peered":"2024-04-25T10:59:18.630462+0000","last_unstale":"2024-04-25T10:59:20.637715+0000","last_undegraded":"2024-04-25T10:59:20.637715+0000","last_fullsized":"2024-04-25T10:59:20.637715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:46.279987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.639078+0000","last_change":"2024-04-25T10:59:19.643805+0000","last_active":"2024-04-25T10:59:20.639078+0000","last_peered":"2024-04-25T10:59:20.639078+0000","last_clean":"2024-04-25T10:59:20.639078+0000","last_became_active":"2024-04-25T10:59:19.643222+0000","last_became_peered":"2024-04-25T10:59:19.643222+0000","last_unstale":"2024-04-25T10:59:20.639078+0000","last_undegraded":"2024-04-25T10:59:20.639078+0000","last_fullsized":"2024-04-25T10:59:20.639078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:07.522883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.185571+0000","last_change":"2024-04-25T10:59:18.627083+0000","last_active":"2024-04-25T10:59:21.185571+0000","last_peered":"2024-04-25T10:59:21.185571+0000","last_clean":"2024-04-25T10:59:21.185571+0000","last_became_active":"2024-04-25T10:59:18.626450+0000","last_became_peered":"2024-04-25T10:59:18.626450+0000","last_unstale":"2024-04-25T10:59:21.185571+0000","last_undegraded":"2024-04-25T10:59:21.185571+0000","last_fullsized":"2024-04-25T10:59:21.185571+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:45.612221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228298+0000","last_change":"2024-04-25T10:59:19.653272+0000","last_active":"2024-04-25T10:59:21.228298+0000","last_peered":"2024-04-25T10:59:21.228298+0000","last_clean":"2024-04-25T10:59:21.228298+0000","last_became_active":"2024-04-25T10:59:19.652954+0000","last_became_peered":"2024-04-25T10:59:19.652954+0000","last_unstale":"2024-04-25T10:59:21.228298+0000","last_undegraded":"2024-04-25T10:59:21.228298+0000","last_fullsized":"2024-04-25T10:59:21.228298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:32.457404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638454+0000","last_change":"2024-04-25T10:59:18.630333+0000","last_active":"2024-04-25T10:59:20.638454+0000","last_peered":"2024-04-25T10:59:20.638454+0000","last_clean":"2024-04-25T10:59:20.638454+0000","last_became_active":"2024-04-25T10:59:18.629185+0000","last_became_peered":"2024-04-25T10:59:18.629185+0000","last_unstale":"2024-04-25T10:59:20.638454+0000","last_undegraded":"2024-04-25T10:59:20.638454+0000","last_fullsized":"2024-04-25T10:59:20.638454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:12.306399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228990+0000","last_change":"2024-04-25T10:59:19.654703+0000","last_active":"2024-04-25T10:59:21.228990+0000","last_peered":"2024-04-25T10:59:21.228990+0000","last_clean":"2024-04-25T10:59:21.228990+0000","last_became_active":"2024-04-25T10:59:19.654381+0000","last_became_peered":"2024-04-25T10:59:19.654381+0000","last_unstale":"2024-04-25T10:59:21.228990+0000","last_undegraded":"2024-04-25T10:59:21.228990+0000","last_fullsized":"2024-04-25T10:59:21.228990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:43.481266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.185751+0000","last_change":"2024-04-25T10:59:18.639591+0000","last_active":"2024-04-25T10:59:21.185751+0000","last_peered":"2024-04-25T10:59:21.185751+0000","last_clean":"2024-04-25T10:59:21.185751+0000","last_became_active":"2024-04-25T10:59:18.639276+0000","last_became_peered":"2024-04-25T10:59:18.639276+0000","last_unstale":"2024-04-25T10:59:21.185751+0000","last_undegraded":"2024-04-25T10:59:21.185751+0000","last_fullsized":"2024-04-25T10:59:21.185751+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:05.044315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.229162+0000","last_change":"2024-04-25T10:59:19.650496+0000","last_active":"2024-04-25T10:59:21.229162+0000","last_peered":"2024-04-25T10:59:21.229162+0000","last_clean":"2024-04-25T10:59:21.229162+0000","last_became_active":"2024-04-25T10:59:19.649725+0000","last_became_peered":"2024-04-25T10:59:19.649725+0000","last_unstale":"2024-04-25T10:59:21.229162+0000","last_undegraded":"2024-04-25T10:59:21.229162+0000","last_fullsized":"2024-04-25T10:59:21.229162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:53.860108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636578+0000","last_change":"2024-04-25T10:59:18.635647+0000","last_active":"2024-04-25T10:59:20.636578+0000","last_peered":"2024-04-25T10:59:20.636578+0000","last_clean":"2024-04-25T10:59:20.636578+0000","last_became_active":"2024-04-25T10:59:18.635264+0000","last_became_peered":"2024-04-25T10:59:18.635264+0000","last_unstale":"2024-04-25T10:59:20.636578+0000","last_undegraded":"2024-04-25T10:59:20.636578+0000","last_fullsized":"2024-04-25T10:59:20.636578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:45.612221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228718+0000","last_change":"2024-04-25T10:59:19.650869+0000","last_active":"2024-04-25T10:59:21.228718+0000","last_peered":"2024-04-25T10:59:21.228718+0000","last_clean":"2024-04-25T10:59:21.228718+0000","last_became_active":"2024-04-25T10:59:19.650541+0000","last_became_peered":"2024-04-25T10:59:19.650541+0000","last_unstale":"2024-04-25T10:59:21.228718+0000","last_undegraded":"2024-04-25T10:59:21.228718+0000","last_fullsized":"2024-04-25T10:59:21.228718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:14.189727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.185690+0000","last_change":"2024-04-25T10:59:18.637436+0000","last_active":"2024-04-25T10:59:21.185690+0000","last_peered":"2024-04-25T10:59:21.185690+0000","last_clean":"2024-04-25T10:59:21.185690+0000","last_became_active":"2024-04-25T10:59:18.637152+0000","last_became_peered":"2024-04-25T10:59:18.637152+0000","last_unstale":"2024-04-25T10:59:21.185690+0000","last_undegraded":"2024-04-25T10:59:21.185690+0000","last_fullsized":"2024-04-25T10:59:21.185690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:10.473815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228358+0000","last_change":"2024-04-25T10:59:19.655023+0000","last_active":"2024-04-25T10:59:21.228358+0000","last_peered":"2024-04-25T10:59:21.228358+0000","last_clean":"2024-04-25T10:59:21.228358+0000","last_became_active":"2024-04-25T10:59:19.654647+0000","last_became_peered":"2024-04-25T10:59:19.654647+0000","last_unstale":"2024-04-25T10:59:21.228358+0000","last_undegraded":"2024-04-25T10:59:21.228358+0000","last_fullsized":"2024-04-25T10:59:21.228358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:00.311029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":110,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227788+0000","last_change":"2024-04-25T10:59:07.839466+0000","last_active":"2024-04-25T10:59:21.227788+0000","last_peered":"2024-04-25T10:59:21.227788+0000","last_clean":"2024-04-25T10:59:21.227788+0000","last_became_active":"2024-04-25T10:59:07.836269+0000","last_became_peered":"2024-04-25T10:59:07.836269+0000","last_unstale":"2024-04-25T10:59:21.227788+0000","last_undegraded":"2024-04-25T10:59:21.227788+0000","last_fullsized":"2024-04-25T10:59:21.227788+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:06.549663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:06.549663+0000","last_clean_scrub_stamp":"2024-04-25T10:59:06.549663+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:21.575330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228265+0000","last_change":"2024-04-25T10:59:18.639570+0000","last_active":"2024-04-25T10:59:21.228265+0000","last_peered":"2024-04-25T10:59:21.228265+0000","last_clean":"2024-04-25T10:59:21.228265+0000","last_became_active":"2024-04-25T10:59:18.639231+0000","last_became_peered":"2024-04-25T10:59:18.639231+0000","last_unstale":"2024-04-25T10:59:21.228265+0000","last_undegraded":"2024-04-25T10:59:21.228265+0000","last_fullsized":"2024-04-25T10:59:21.228265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:28.594312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228305+0000","last_change":"2024-04-25T10:59:19.652991+0000","last_active":"2024-04-25T10:59:21.228305+0000","last_peered":"2024-04-25T10:59:21.228305+0000","last_clean":"2024-04-25T10:59:21.228305+0000","last_became_active":"2024-04-25T10:59:19.652680+0000","last_became_peered":"2024-04-25T10:59:19.652680+0000","last_unstale":"2024-04-25T10:59:21.228305+0000","last_undegraded":"2024-04-25T10:59:21.228305+0000","last_fullsized":"2024-04-25T10:59:21.228305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:38.297267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228139+0000","last_change":"2024-04-25T10:59:18.639571+0000","last_active":"2024-04-25T10:59:21.228139+0000","last_peered":"2024-04-25T10:59:21.228139+0000","last_clean":"2024-04-25T10:59:21.228139+0000","last_became_active":"2024-04-25T10:59:18.639065+0000","last_became_peered":"2024-04-25T10:59:18.639065+0000","last_unstale":"2024-04-25T10:59:21.228139+0000","last_undegraded":"2024-04-25T10:59:21.228139+0000","last_fullsized":"2024-04-25T10:59:21.228139+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:10.473815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227726+0000","last_change":"2024-04-25T10:59:19.644241+0000","last_active":"2024-04-25T10:59:21.227726+0000","last_peered":"2024-04-25T10:59:21.227726+0000","last_clean":"2024-04-25T10:59:21.227726+0000","last_became_active":"2024-04-25T10:59:19.643778+0000","last_became_peered":"2024-04-25T10:59:19.643778+0000","last_unstale":"2024-04-25T10:59:21.227726+0000","last_undegraded":"2024-04-25T10:59:21.227726+0000","last_fullsized":"2024-04-25T10:59:21.227726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:10.276726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.185642+0000","last_change":"2024-04-25T10:59:18.643811+0000","last_active":"2024-04-25T10:59:21.185642+0000","last_peered":"2024-04-25T10:59:21.185642+0000","last_clean":"2024-04-25T10:59:21.185642+0000","last_became_active":"2024-04-25T10:59:18.643454+0000","last_became_peered":"2024-04-25T10:59:18.643454+0000","last_unstale":"2024-04-25T10:59:21.185642+0000","last_undegraded":"2024-04-25T10:59:21.185642+0000","last_fullsized":"2024-04-25T10:59:21.185642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:09.933183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636785+0000","last_change":"2024-04-25T10:59:19.639259+0000","last_active":"2024-04-25T10:59:20.636785+0000","last_peered":"2024-04-25T10:59:20.636785+0000","last_clean":"2024-04-25T10:59:20.636785+0000","last_became_active":"2024-04-25T10:59:19.637680+0000","last_became_peered":"2024-04-25T10:59:19.637680+0000","last_unstale":"2024-04-25T10:59:20.636785+0000","last_undegraded":"2024-04-25T10:59:20.636785+0000","last_fullsized":"2024-04-25T10:59:20.636785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:33.050282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.229027+0000","last_change":"2024-04-25T10:59:18.643664+0000","last_active":"2024-04-25T10:59:21.229027+0000","last_peered":"2024-04-25T10:59:21.229027+0000","last_clean":"2024-04-25T10:59:21.229027+0000","last_became_active":"2024-04-25T10:59:18.643309+0000","last_became_peered":"2024-04-25T10:59:18.643309+0000","last_unstale":"2024-04-25T10:59:21.229027+0000","last_undegraded":"2024-04-25T10:59:21.229027+0000","last_fullsized":"2024-04-25T10:59:21.229027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:45.612221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.639084+0000","last_change":"2024-04-25T10:59:19.633507+0000","last_active":"2024-04-25T10:59:20.639084+0000","last_peered":"2024-04-25T10:59:20.639084+0000","last_clean":"2024-04-25T10:59:20.639084+0000","last_became_active":"2024-04-25T10:59:19.633182+0000","last_became_peered":"2024-04-25T10:59:19.633182+0000","last_unstale":"2024-04-25T10:59:20.639084+0000","last_undegraded":"2024-04-25T10:59:20.639084+0000","last_fullsized":"2024-04-25T10:59:20.639084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:11.481375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636672+0000","last_change":"2024-04-25T10:59:18.635772+0000","last_active":"2024-04-25T10:59:20.636672+0000","last_peered":"2024-04-25T10:59:20.636672+0000","last_clean":"2024-04-25T10:59:20.636672+0000","last_became_active":"2024-04-25T10:59:18.635486+0000","last_became_peered":"2024-04-25T10:59:18.635486+0000","last_unstale":"2024-04-25T10:59:20.636672+0000","last_undegraded":"2024-04-25T10:59:20.636672+0000","last_fullsized":"2024-04-25T10:59:20.636672+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:24.267022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228031+0000","last_change":"2024-04-25T10:59:19.648239+0000","last_active":"2024-04-25T10:59:21.228031+0000","last_peered":"2024-04-25T10:59:21.228031+0000","last_clean":"2024-04-25T10:59:21.228031+0000","last_became_active":"2024-04-25T10:59:19.646876+0000","last_became_peered":"2024-04-25T10:59:19.646876+0000","last_unstale":"2024-04-25T10:59:21.228031+0000","last_undegraded":"2024-04-25T10:59:21.228031+0000","last_fullsized":"2024-04-25T10:59:21.228031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:49.314956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228833+0000","last_change":"2024-04-25T10:59:18.643449+0000","last_active":"2024-04-25T10:59:21.228833+0000","last_peered":"2024-04-25T10:59:21.228833+0000","last_clean":"2024-04-25T10:59:21.228833+0000","last_became_active":"2024-04-25T10:59:18.643069+0000","last_became_peered":"2024-04-25T10:59:18.643069+0000","last_unstale":"2024-04-25T10:59:21.228833+0000","last_undegraded":"2024-04-25T10:59:21.228833+0000","last_fullsized":"2024-04-25T10:59:21.228833+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:57.176546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636600+0000","last_change":"2024-04-25T10:59:19.638854+0000","last_active":"2024-04-25T10:59:20.636600+0000","last_peered":"2024-04-25T10:59:20.636600+0000","last_clean":"2024-04-25T10:59:20.636600+0000","last_became_active":"2024-04-25T10:59:19.638273+0000","last_became_peered":"2024-04-25T10:59:19.638273+0000","last_unstale":"2024-04-25T10:59:20.636600+0000","last_undegraded":"2024-04-25T10:59:20.636600+0000","last_fullsized":"2024-04-25T10:59:20.636600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:15.132862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.428925+0000","last_change":"2024-04-25T10:59:18.635651+0000","last_active":"2024-04-25T10:59:21.428925+0000","last_peered":"2024-04-25T10:59:21.428925+0000","last_clean":"2024-04-25T10:59:21.428925+0000","last_became_active":"2024-04-25T10:59:18.635348+0000","last_became_peered":"2024-04-25T10:59:18.635348+0000","last_unstale":"2024-04-25T10:59:21.428925+0000","last_undegraded":"2024-04-25T10:59:21.428925+0000","last_fullsized":"2024-04-25T10:59:21.428925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:58.505038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228269+0000","last_change":"2024-04-25T10:59:19.640663+0000","last_active":"2024-04-25T10:59:21.228269+0000","last_peered":"2024-04-25T10:59:21.228269+0000","last_clean":"2024-04-25T10:59:21.228269+0000","last_became_active":"2024-04-25T10:59:19.640311+0000","last_became_peered":"2024-04-25T10:59:19.640311+0000","last_unstale":"2024-04-25T10:59:21.228269+0000","last_undegraded":"2024-04-25T10:59:21.228269+0000","last_fullsized":"2024-04-25T10:59:21.228269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:57.344787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638534+0000","last_change":"2024-04-25T10:59:18.635706+0000","last_active":"2024-04-25T10:59:20.638534+0000","last_peered":"2024-04-25T10:59:20.638534+0000","last_clean":"2024-04-25T10:59:20.638534+0000","last_became_active":"2024-04-25T10:59:18.635377+0000","last_became_peered":"2024-04-25T10:59:18.635377+0000","last_unstale":"2024-04-25T10:59:20.638534+0000","last_undegraded":"2024-04-25T10:59:20.638534+0000","last_fullsized":"2024-04-25T10:59:20.638534+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:24.267022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227935+0000","last_change":"2024-04-25T10:59:19.635255+0000","last_active":"2024-04-25T10:59:21.227935+0000","last_peered":"2024-04-25T10:59:21.227935+0000","last_clean":"2024-04-25T10:59:21.227935+0000","last_became_active":"2024-04-25T10:59:19.634829+0000","last_became_peered":"2024-04-25T10:59:19.634829+0000","last_unstale":"2024-04-25T10:59:21.227935+0000","last_undegraded":"2024-04-25T10:59:21.227935+0000","last_fullsized":"2024-04-25T10:59:21.227935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:52.815596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227702+0000","last_change":"2024-04-25T10:59:18.643806+0000","last_active":"2024-04-25T10:59:21.227702+0000","last_peered":"2024-04-25T10:59:21.227702+0000","last_clean":"2024-04-25T10:59:21.227702+0000","last_became_active":"2024-04-25T10:59:18.643374+0000","last_became_peered":"2024-04-25T10:59:18.643374+0000","last_unstale":"2024-04-25T10:59:21.227702+0000","last_undegraded":"2024-04-25T10:59:21.227702+0000","last_fullsized":"2024-04-25T10:59:21.227702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:45.612221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.637981+0000","last_change":"2024-04-25T10:59:19.637663+0000","last_active":"2024-04-25T10:59:20.637981+0000","last_peered":"2024-04-25T10:59:20.637981+0000","last_clean":"2024-04-25T10:59:20.637981+0000","last_became_active":"2024-04-25T10:59:19.637309+0000","last_became_peered":"2024-04-25T10:59:19.637309+0000","last_unstale":"2024-04-25T10:59:20.637981+0000","last_undegraded":"2024-04-25T10:59:20.637981+0000","last_fullsized":"2024-04-25T10:59:20.637981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:29.601873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636484+0000","last_change":"2024-04-25T10:59:18.636340+0000","last_active":"2024-04-25T10:59:20.636484+0000","last_peered":"2024-04-25T10:59:20.636484+0000","last_clean":"2024-04-25T10:59:20.636484+0000","last_became_active":"2024-04-25T10:59:18.636036+0000","last_became_peered":"2024-04-25T10:59:18.636036+0000","last_unstale":"2024-04-25T10:59:20.636484+0000","last_undegraded":"2024-04-25T10:59:20.636484+0000","last_fullsized":"2024-04-25T10:59:20.636484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:15.754476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227666+0000","last_change":"2024-04-25T10:59:19.632935+0000","last_active":"2024-04-25T10:59:21.227666+0000","last_peered":"2024-04-25T10:59:21.227666+0000","last_clean":"2024-04-25T10:59:21.227666+0000","last_became_active":"2024-04-25T10:59:19.632658+0000","last_became_peered":"2024-04-25T10:59:19.632658+0000","last_unstale":"2024-04-25T10:59:21.227666+0000","last_undegraded":"2024-04-25T10:59:21.227666+0000","last_fullsized":"2024-04-25T10:59:21.227666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:52.815596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.636560+0000","last_change":"2024-04-25T10:59:18.639304+0000","last_active":"2024-04-25T10:59:20.636560+0000","last_peered":"2024-04-25T10:59:20.636560+0000","last_clean":"2024-04-25T10:59:20.636560+0000","last_became_active":"2024-04-25T10:59:18.638939+0000","last_became_peered":"2024-04-25T10:59:18.638939+0000","last_unstale":"2024-04-25T10:59:20.636560+0000","last_undegraded":"2024-04-25T10:59:20.636560+0000","last_fullsized":"2024-04-25T10:59:20.636560+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:28.594312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227866+0000","last_change":"2024-04-25T10:59:19.631658+0000","last_active":"2024-04-25T10:59:21.227866+0000","last_peered":"2024-04-25T10:59:21.227866+0000","last_clean":"2024-04-25T10:59:21.227866+0000","last_became_active":"2024-04-25T10:59:19.631363+0000","last_became_peered":"2024-04-25T10:59:19.631363+0000","last_unstale":"2024-04-25T10:59:21.227866+0000","last_undegraded":"2024-04-25T10:59:21.227866+0000","last_fullsized":"2024-04-25T10:59:21.227866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:33.050282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228855+0000","last_change":"2024-04-25T10:59:18.638514+0000","last_active":"2024-04-25T10:59:21.228855+0000","last_peered":"2024-04-25T10:59:21.228855+0000","last_clean":"2024-04-25T10:59:21.228855+0000","last_became_active":"2024-04-25T10:59:18.638113+0000","last_became_peered":"2024-04-25T10:59:18.638113+0000","last_unstale":"2024-04-25T10:59:21.228855+0000","last_undegraded":"2024-04-25T10:59:21.228855+0000","last_fullsized":"2024-04-25T10:59:21.228855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:09.933183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:20.638752+0000","last_change":"2024-04-25T10:59:19.652586+0000","last_active":"2024-04-25T10:59:20.638752+0000","last_peered":"2024-04-25T10:59:20.638752+0000","last_clean":"2024-04-25T10:59:20.638752+0000","last_became_active":"2024-04-25T10:59:19.652246+0000","last_became_peered":"2024-04-25T10:59:19.652246+0000","last_unstale":"2024-04-25T10:59:20.638752+0000","last_undegraded":"2024-04-25T10:59:20.638752+0000","last_fullsized":"2024-04-25T10:59:20.638752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:52.815596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228042+0000","last_change":"2024-04-25T10:59:19.649474+0000","last_active":"2024-04-25T10:59:21.228042+0000","last_peered":"2024-04-25T10:59:21.228042+0000","last_clean":"2024-04-25T10:59:21.228042+0000","last_became_active":"2024-04-25T10:59:19.649134+0000","last_became_peered":"2024-04-25T10:59:19.649134+0000","last_unstale":"2024-04-25T10:59:21.228042+0000","last_undegraded":"2024-04-25T10:59:21.228042+0000","last_fullsized":"2024-04-25T10:59:21.228042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:38.297267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228094+0000","last_change":"2024-04-25T10:59:18.641120+0000","last_active":"2024-04-25T10:59:21.228094+0000","last_peered":"2024-04-25T10:59:21.228094+0000","last_clean":"2024-04-25T10:59:21.228094+0000","last_became_active":"2024-04-25T10:59:18.640791+0000","last_became_peered":"2024-04-25T10:59:18.640791+0000","last_unstale":"2024-04-25T10:59:21.228094+0000","last_undegraded":"2024-04-25T10:59:21.228094+0000","last_fullsized":"2024-04-25T10:59:21.228094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:43.470493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228726+0000","last_change":"2024-04-25T10:59:19.654395+0000","last_active":"2024-04-25T10:59:21.228726+0000","last_peered":"2024-04-25T10:59:21.228726+0000","last_clean":"2024-04-25T10:59:21.228726+0000","last_became_active":"2024-04-25T10:59:19.654107+0000","last_became_peered":"2024-04-25T10:59:19.654107+0000","last_unstale":"2024-04-25T10:59:21.228726+0000","last_undegraded":"2024-04-25T10:59:21.228726+0000","last_fullsized":"2024-04-25T10:59:21.228726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:33.154184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228501+0000","last_change":"2024-04-25T10:59:18.640843+0000","last_active":"2024-04-25T10:59:21.228501+0000","last_peered":"2024-04-25T10:59:21.228501+0000","last_clean":"2024-04-25T10:59:21.228501+0000","last_became_active":"2024-04-25T10:59:18.640496+0000","last_became_peered":"2024-04-25T10:59:18.640496+0000","last_unstale":"2024-04-25T10:59:21.228501+0000","last_undegraded":"2024-04-25T10:59:21.228501+0000","last_fullsized":"2024-04-25T10:59:21.228501+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:05.044315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.227754+0000","last_change":"2024-04-25T10:59:19.652337+0000","last_active":"2024-04-25T10:59:21.227754+0000","last_peered":"2024-04-25T10:59:21.227754+0000","last_clean":"2024-04-25T10:59:21.227754+0000","last_became_active":"2024-04-25T10:59:19.652008+0000","last_became_peered":"2024-04-25T10:59:19.652008+0000","last_unstale":"2024-04-25T10:59:21.227754+0000","last_undegraded":"2024-04-25T10:59:21.227754+0000","last_fullsized":"2024-04-25T10:59:21.227754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:32:26.553467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228556+0000","last_change":"2024-04-25T10:59:18.643508+0000","last_active":"2024-04-25T10:59:21.228556+0000","last_peered":"2024-04-25T10:59:21.228556+0000","last_clean":"2024-04-25T10:59:21.228556+0000","last_became_active":"2024-04-25T10:59:18.643186+0000","last_became_peered":"2024-04-25T10:59:18.643186+0000","last_unstale":"2024-04-25T10:59:21.228556+0000","last_undegraded":"2024-04-25T10:59:21.228556+0000","last_fullsized":"2024-04-25T10:59:21.228556+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:35.341389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228500+0000","last_change":"2024-04-25T10:59:19.640778+0000","last_active":"2024-04-25T10:59:21.228500+0000","last_peered":"2024-04-25T10:59:21.228500+0000","last_clean":"2024-04-25T10:59:21.228500+0000","last_became_active":"2024-04-25T10:59:19.640479+0000","last_became_peered":"2024-04-25T10:59:19.640479+0000","last_unstale":"2024-04-25T10:59:21.228500+0000","last_undegraded":"2024-04-25T10:59:21.228500+0000","last_fullsized":"2024-04-25T10:59:21.228500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:18.609520+0000","last_clean_scrub_stamp":"2024-04-25T10:59:18.609520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:52.334437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:59:21.228246+0000","last_change":"2024-04-25T10:59:18.635569+0000","last_active":"2024-04-25T10:59:21.228246+0000","last_peered":"2024-04-25T10:59:21.228246+0000","last_clean":"2024-04-25T10:59:21.228246+0000","last_became_active":"2024-04-25T10:59:18.635303+0000","last_became_peered":"2024-04-25T10:59:18.635303+0000","last_unstale":"2024-04-25T10:59:21.228246+0000","last_undegraded":"2024-04-25T10:59:21.228246+0000","last_fullsized":"2024-04-25T10:59:21.228246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:17.601959+0000","last_clean_scrub_stamp":"2024-04-25T10:59:17.601959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:46.279987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:59:25.794 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T10:59:25.794 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T10:59:25.794 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T10:59:25.794 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:59:25.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.936+0000 7f443f184640 1 Processor -- start 2024-04-25T10:59:25.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.937+0000 7f443f184640 1 -- start start 2024-04-25T10:59:25.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.938+0000 7f443f184640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44380b2cc0 0x7f44380b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.938+0000 7f443f184640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f44380d3860 0x7f44380d3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.938+0000 7f443f184640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44380d41b0 con 0x7f44380b9160 2024-04-25T10:59:25.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.938+0000 7f443f184640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f44380b4ab0 con 0x7f44380d3860 2024-04-25T10:59:25.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.938+0000 7f443f184640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f44380b4180 con 0x7f44380b2cc0 2024-04-25T10:59:25.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.938+0000 7f443d721640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f44380d3860 0x7f44380d3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.938+0000 7f4437fff640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f44380b9160 0x7f44380d27e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:57136/0 (socket says 172.21.15.69:57136) 2024-04-25T10:59:25.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.938+0000 7f4437fff640 1 -- 172.21.15.69:0/3537610492 learned_addr learned my addr 172.21.15.69:0/3537610492 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:25.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.938+0000 7f443cf20640 1 --2- 172.21.15.69:0/3537610492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44380b2cc0 0x7f44380b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.938+0000 7f443d721640 1 -- 172.21.15.69:0/3537610492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44380b2cc0 msgr2=0x7f44380b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:25.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.938+0000 7f443d721640 1 --2- 172.21.15.69:0/3537610492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44380b2cc0 0x7f44380b4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.938+0000 7f443d721640 1 -- 172.21.15.69:0/3537610492 >> v1:172.21.15.69:6789/0 conn(0x7f44380b9160 legacy=0x7f44380d27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.938+0000 7f443d721640 1 -- 172.21.15.69:0/3537610492 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44380b4460 con 0x7f44380d3860 2024-04-25T10:59:25.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.938+0000 7f44377fe640 1 -- 172.21.15.69:0/3537610492 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 691349164 0 0) 0x7f4420001110 con 0x7f44380b9160 2024-04-25T10:59:25.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.939+0000 7f443d721640 1 --2- 172.21.15.69:0/3537610492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44380d3860 0x7f44380d3c70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f44280097a0 tx=0x7f442802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382f62e27d4d39e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:25.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.939+0000 7f44377fe640 1 -- 172.21.15.69:0/3537610492 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4428037070 con 0x7f44380d3860 2024-04-25T10:59:25.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.939+0000 7f44377fe640 1 -- 172.21.15.69:0/3537610492 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f442802ae50 con 0x7f44380d3860 2024-04-25T10:59:25.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.939+0000 7f44377fe640 1 -- 172.21.15.69:0/3537610492 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4428032660 con 0x7f44380d3860 2024-04-25T10:59:25.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.940+0000 7f443f184640 1 -- 172.21.15.69:0/3537610492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44380d3860 msgr2=0x7f44380d3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.940+0000 7f443f184640 1 --2- 172.21.15.69:0/3537610492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44380d3860 0x7f44380d3c70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f44280097a0 tx=0x7f442802a100 comp rx=0 tx=0).stop 2024-04-25T10:59:25.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.940+0000 7f443f184640 1 -- 172.21.15.69:0/3537610492 shutdown_connections 2024-04-25T10:59:25.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.940+0000 7f443f184640 1 --2- 172.21.15.69:0/3537610492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44380d3860 0x7f44380d3c70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.940+0000 7f443f184640 1 --2- 172.21.15.69:0/3537610492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44380b2cc0 0x7f44380b4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.940+0000 7f443f184640 1 -- 172.21.15.69:0/3537610492 >> 172.21.15.69:0/3537610492 conn(0x7f443808d110 msgr2=0x7f443808d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:25.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.940+0000 7f443f184640 1 -- 172.21.15.69:0/3537610492 shutdown_connections 2024-04-25T10:59:25.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.940+0000 7f443f184640 1 -- 172.21.15.69:0/3537610492 wait complete. 2024-04-25T10:59:25.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.942+0000 7f443f184640 1 Processor -- start 2024-04-25T10:59:25.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.942+0000 7f443f184640 1 -- start start 2024-04-25T10:59:25.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.943+0000 7f443f184640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44380b2cc0 0x7f44381521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.943+0000 7f443f184640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44380b9160 0x7f4438160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.943+0000 7f443f184640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44380d3860 0x7f44380cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.943+0000 7f443f184640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f44380b05c0 con 0x7f44380d3860 2024-04-25T10:59:25.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.943+0000 7f443f184640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f44380b9570 con 0x7f44380b9160 2024-04-25T10:59:25.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.943+0000 7f443f184640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f44380b0740 con 0x7f44380b2cc0 2024-04-25T10:59:25.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.943+0000 7f443d721640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44380d3860 0x7f44380cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.943+0000 7f443d721640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44380d3860 0x7f44380cf2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56266/0 (socket says 172.21.15.69:56266) 2024-04-25T10:59:25.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.943+0000 7f443d721640 1 -- 172.21.15.69:0/1113480714 learned_addr learned my addr 172.21.15.69:0/1113480714 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:25.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.944+0000 7f443cf20640 1 --2- 172.21.15.69:0/1113480714 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44380b2cc0 0x7f44381521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.944+0000 7f4437fff640 1 --2- 172.21.15.69:0/1113480714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44380b9160 0x7f4438160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.944+0000 7f443d721640 1 -- 172.21.15.69:0/1113480714 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44380b2cc0 msgr2=0x7f44381521b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.944+0000 7f443d721640 1 --2- 172.21.15.69:0/1113480714 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44380b2cc0 0x7f44381521b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.944+0000 7f443d721640 1 -- 172.21.15.69:0/1113480714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44380b9160 msgr2=0x7f4438160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:25.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.944+0000 7f443d721640 1 --2- 172.21.15.69:0/1113480714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44380b9160 0x7f4438160ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:25.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.944+0000 7f443d721640 1 -- 172.21.15.69:0/1113480714 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44380cabb0 con 0x7f44380d3860 2024-04-25T10:59:25.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.944+0000 7f443cf20640 1 --2- 172.21.15.69:0/1113480714 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44380b2cc0 0x7f44381521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:25.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.944+0000 7f4437fff640 1 --2- 172.21.15.69:0/1113480714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44380b9160 0x7f4438160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:25.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.944+0000 7f443d721640 1 --2- 172.21.15.69:0/1113480714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44380d3860 0x7f44380cf2a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4428035040 tx=0x7f44280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:25.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.944+0000 7f44357fa640 1 -- 172.21.15.69:0/1113480714 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4428037040 con 0x7f44380d3860 2024-04-25T10:59:25.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.944+0000 7f44357fa640 1 -- 172.21.15.69:0/1113480714 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4428004030 con 0x7f44380d3860 2024-04-25T10:59:25.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.944+0000 7f443f184640 1 -- 172.21.15.69:0/1113480714 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44380cae90 con 0x7f44380d3860 2024-04-25T10:59:25.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.944+0000 7f44357fa640 1 -- 172.21.15.69:0/1113480714 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4428006e10 con 0x7f44380d3860 2024-04-25T10:59:25.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.944+0000 7f443f184640 1 -- 172.21.15.69:0/1113480714 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44380cb450 con 0x7f44380d3860 2024-04-25T10:59:25.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.945+0000 7f443f184640 1 -- 172.21.15.69:0/1113480714 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4438050740 con 0x7f44380d3860 2024-04-25T10:59:25.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.946+0000 7f44357fa640 1 -- 172.21.15.69:0/1113480714 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4428003610 con 0x7f44380d3860 2024-04-25T10:59:25.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.946+0000 7f44357fa640 1 --2- 172.21.15.69:0/1113480714 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f44180744b0 0x7f44180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:25.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.946+0000 7f44357fa640 1 -- 172.21.15.69:0/1113480714 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f44280b5800 con 0x7f44380d3860 2024-04-25T10:59:25.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.946+0000 7f443cf20640 1 --2- 172.21.15.69:0/1113480714 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f44180744b0 0x7f44180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:25.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.947+0000 7f443cf20640 1 --2- 172.21.15.69:0/1113480714 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f44180744b0 0x7f44180769c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4430007440 tx=0x7f443000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:25.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:25.951+0000 7f44357fa640 1 -- 172.21.15.69:0/1113480714 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4428042020 con 0x7f44380d3860 2024-04-25T10:59:26.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.162+0000 7f443f184640 1 -- 172.21.15.69:0/1113480714 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4438076240 con 0x7f44380d3860 2024-04-25T10:59:26.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.163+0000 7f44357fa640 1 -- 172.21.15.69:0/1113480714 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f4428080020 con 0x7f44380d3860 2024-04-25T10:59:26.163 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:26.163 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:59:26.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.164+0000 7f443f184640 1 -- 172.21.15.69:0/1113480714 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f44180744b0 msgr2=0x7f44180769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:26.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.164+0000 7f443f184640 1 --2- 172.21.15.69:0/1113480714 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f44180744b0 0x7f44180769c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4430007440 tx=0x7f443000b040 comp rx=0 tx=0).stop 2024-04-25T10:59:26.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.164+0000 7f443f184640 1 -- 172.21.15.69:0/1113480714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44380d3860 msgr2=0x7f44380cf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:26.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.164+0000 7f443f184640 1 --2- 172.21.15.69:0/1113480714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44380d3860 0x7f44380cf2a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4428035040 tx=0x7f44280046c0 comp rx=0 tx=0).stop 2024-04-25T10:59:26.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.164+0000 7f443f184640 1 -- 172.21.15.69:0/1113480714 shutdown_connections 2024-04-25T10:59:26.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.164+0000 7f443f184640 1 --2- 172.21.15.69:0/1113480714 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f44180744b0 0x7f44180769c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:26.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.164+0000 7f443f184640 1 --2- 172.21.15.69:0/1113480714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44380d3860 0x7f44380cf2a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:26.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.164+0000 7f443f184640 1 --2- 172.21.15.69:0/1113480714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44380b9160 0x7f4438160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:26.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.164+0000 7f443f184640 1 --2- 172.21.15.69:0/1113480714 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44380b2cc0 0x7f44381521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:26.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.164+0000 7f443f184640 1 -- 172.21.15.69:0/1113480714 >> 172.21.15.69:0/1113480714 conn(0x7f443808d110 msgr2=0x7f44381600f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:26.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.165+0000 7f443f184640 1 -- 172.21.15.69:0/1113480714 shutdown_connections 2024-04-25T10:59:26.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.165+0000 7f443f184640 1 -- 172.21.15.69:0/1113480714 wait complete. 2024-04-25T10:59:26.176 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T10:59:26.177 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:59:26.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.283+0000 7f08ab30d640 1 Processor -- start 2024-04-25T10:59:26.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.284+0000 7f08ab30d640 1 -- start start 2024-04-25T10:59:26.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.285+0000 7f08ab30d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08a40b7750 0x7f08a40d1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:26.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.285+0000 7f08ab30d640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f08a406d7e0 0x7f08a406dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:26.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.285+0000 7f08ab30d640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08a406e130 con 0x7f08a40cf610 2024-04-25T10:59:26.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.285+0000 7f08ab30d640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f08a40b3af0 con 0x7f08a406d7e0 2024-04-25T10:59:26.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.285+0000 7f08ab30d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f08a4164320 con 0x7f08a40b7750 2024-04-25T10:59:26.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.285+0000 7f08a98aa640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f08a406d7e0 0x7f08a406dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:26.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.285+0000 7f08a88a8640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f08a40cf610 0x7f08a4164800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:57148/0 (socket says 172.21.15.69:57148) 2024-04-25T10:59:26.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.285+0000 7f08a98aa640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f08a406d7e0 0x7f08a406dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56272/0 (socket says 172.21.15.69:56272) 2024-04-25T10:59:26.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.285+0000 7f08a88a8640 1 -- 172.21.15.69:0/201241051 learned_addr learned my addr 172.21.15.69:0/201241051 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:26.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.285+0000 7f08a90a9640 1 --2- 172.21.15.69:0/201241051 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08a40b7750 0x7f08a40d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:26.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.285+0000 7f08a98aa640 1 -- 172.21.15.69:0/201241051 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08a40b7750 msgr2=0x7f08a40d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:26.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.285+0000 7f08a98aa640 1 --2- 172.21.15.69:0/201241051 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08a40b7750 0x7f08a40d1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:26.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.285+0000 7f08a98aa640 1 -- 172.21.15.69:0/201241051 >> v1:172.21.15.69:6789/0 conn(0x7f08a40cf610 legacy=0x7f08a4164800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:26.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.285+0000 7f08a98aa640 1 -- 172.21.15.69:0/201241051 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a40b2f70 con 0x7f08a406d7e0 2024-04-25T10:59:26.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.285+0000 7f089bfff640 1 -- 172.21.15.69:0/201241051 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2397979501 0 0) 0x7f0894001110 con 0x7f08a40cf610 2024-04-25T10:59:26.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.286+0000 7f08a98aa640 1 --2- 172.21.15.69:0/201241051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08a406d7e0 0x7f08a406dbf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f089c009d90 tx=0x7f089c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b59aef3cc96ee2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:26.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.286+0000 7f089bfff640 1 -- 172.21.15.69:0/201241051 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089c031600 con 0x7f08a406d7e0 2024-04-25T10:59:26.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.286+0000 7f089bfff640 1 -- 172.21.15.69:0/201241051 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f089c0317a0 con 0x7f08a406d7e0 2024-04-25T10:59:26.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.286+0000 7f089bfff640 1 -- 172.21.15.69:0/201241051 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089c031b70 con 0x7f08a406d7e0 2024-04-25T10:59:26.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.287+0000 7f08ab30d640 1 -- 172.21.15.69:0/201241051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08a406d7e0 msgr2=0x7f08a406dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:26.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.287+0000 7f08ab30d640 1 --2- 172.21.15.69:0/201241051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08a406d7e0 0x7f08a406dbf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f089c009d90 tx=0x7f089c02a5d0 comp rx=0 tx=0).stop 2024-04-25T10:59:26.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.287+0000 7f08ab30d640 1 -- 172.21.15.69:0/201241051 shutdown_connections 2024-04-25T10:59:26.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.287+0000 7f08ab30d640 1 --2- 172.21.15.69:0/201241051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08a406d7e0 0x7f08a406dbf0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:26.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.287+0000 7f08ab30d640 1 --2- 172.21.15.69:0/201241051 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08a40b7750 0x7f08a40d1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:26.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.287+0000 7f08ab30d640 1 -- 172.21.15.69:0/201241051 >> 172.21.15.69:0/201241051 conn(0x7f08a409b7d0 msgr2=0x7f08a40b7040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:26.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.287+0000 7f08ab30d640 1 -- 172.21.15.69:0/201241051 shutdown_connections 2024-04-25T10:59:26.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.287+0000 7f08ab30d640 1 -- 172.21.15.69:0/201241051 wait complete. 2024-04-25T10:59:26.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.289+0000 7f08ab30d640 1 Processor -- start 2024-04-25T10:59:26.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.289+0000 7f08ab30d640 1 -- start start 2024-04-25T10:59:26.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.290+0000 7f08ab30d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08a406d7e0 0x7f08a409caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:26.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.290+0000 7f08ab30d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08a40b7750 0x7f08a40d1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:26.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.290+0000 7f08ab30d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08a40cf610 0x7f08a40d1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:26.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.290+0000 7f08ab30d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f08a40b3810 con 0x7f08a406d7e0 2024-04-25T10:59:26.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.290+0000 7f08ab30d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f08a40b3530 con 0x7f08a40cf610 2024-04-25T10:59:26.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.290+0000 7f08ab30d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f08a40cfa20 con 0x7f08a40b7750 2024-04-25T10:59:26.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.290+0000 7f08a90a9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08a406d7e0 0x7f08a409caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:26.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.290+0000 7f08a90a9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08a406d7e0 0x7f08a409caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56284/0 (socket says 172.21.15.69:56284) 2024-04-25T10:59:26.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.290+0000 7f08a90a9640 1 -- 172.21.15.69:0/766335835 learned_addr learned my addr 172.21.15.69:0/766335835 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:26.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.290+0000 7f08a88a8640 1 --2- 172.21.15.69:0/766335835 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08a40b7750 0x7f08a40d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:26.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.290+0000 7f08a90a9640 1 -- 172.21.15.69:0/766335835 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08a40b7750 msgr2=0x7f08a40d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:26.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.290+0000 7f08a90a9640 1 --2- 172.21.15.69:0/766335835 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08a40b7750 0x7f08a40d1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:26.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.290+0000 7f08a90a9640 1 -- 172.21.15.69:0/766335835 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08a40cf610 msgr2=0x7f08a40d1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:26.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.290+0000 7f08a90a9640 1 --2- 172.21.15.69:0/766335835 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08a40cf610 0x7f08a40d1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:26.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.290+0000 7f08a90a9640 1 -- 172.21.15.69:0/766335835 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a414f620 con 0x7f08a406d7e0 2024-04-25T10:59:26.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.290+0000 7f08a88a8640 1 --2- 172.21.15.69:0/766335835 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08a40b7750 0x7f08a40d1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:26.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.291+0000 7f08a90a9640 1 --2- 172.21.15.69:0/766335835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08a406d7e0 0x7f08a409caa0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f08a000e870 tx=0x7f08a000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:26.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.291+0000 7f0899ffb640 1 -- 172.21.15.69:0/766335835 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a0017070 con 0x7f08a406d7e0 2024-04-25T10:59:26.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.291+0000 7f0899ffb640 1 -- 172.21.15.69:0/766335835 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08a000ee60 con 0x7f08a406d7e0 2024-04-25T10:59:26.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.291+0000 7f0899ffb640 1 -- 172.21.15.69:0/766335835 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a000cb40 con 0x7f08a406d7e0 2024-04-25T10:59:26.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.291+0000 7f08ab30d640 1 -- 172.21.15.69:0/766335835 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08a414f900 con 0x7f08a406d7e0 2024-04-25T10:59:26.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.291+0000 7f08ab30d640 1 -- 172.21.15.69:0/766335835 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08a414fec0 con 0x7f08a406d7e0 2024-04-25T10:59:26.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.292+0000 7f08ab30d640 1 -- 172.21.15.69:0/766335835 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08a4050740 con 0x7f08a406d7e0 2024-04-25T10:59:26.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.293+0000 7f0899ffb640 1 -- 172.21.15.69:0/766335835 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f08a001a070 con 0x7f08a406d7e0 2024-04-25T10:59:26.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.293+0000 7f0899ffb640 1 --2- 172.21.15.69:0/766335835 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f08880744b0 0x7f08880769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:26.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.293+0000 7f0899ffb640 1 -- 172.21.15.69:0/766335835 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f08a00957e0 con 0x7f08a406d7e0 2024-04-25T10:59:26.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.293+0000 7f08a88a8640 1 --2- 172.21.15.69:0/766335835 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f08880744b0 0x7f08880769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:26.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.294+0000 7f08a88a8640 1 --2- 172.21.15.69:0/766335835 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f08880744b0 0x7f08880769c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f08a40d23d0 tx=0x7f0894009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:26.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.298+0000 7f0899ffb640 1 -- 172.21.15.69:0/766335835 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f08a005ff70 con 0x7f08a406d7e0 2024-04-25T10:59:26.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.474+0000 7f08ab30d640 1 -- 172.21.15.69:0/766335835 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f08a40d2f00 con 0x7f08a406d7e0 2024-04-25T10:59:26.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.475+0000 7f0899ffb640 1 -- 172.21.15.69:0/766335835 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f08a005f870 con 0x7f08a406d7e0 2024-04-25T10:59:26.476 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:26.476 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":20,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T10:59:20.630151+0000","last_up_change":"2024-04-25T10:59:05.546839+0000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:59:06.422674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:59:16.727980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:59:17.938486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6825","nonce":1876101065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6827","nonce":1876101065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6831","nonce":1876101065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6829","nonce":1876101065}]},"public_addr":"172.21.15.69:6825/1876101065","cluster_addr":"172.21.15.69:6827/1876101065","heartbeat_back_addr":"172.21.15.69:6831/1876101065","heartbeat_front_addr":"172.21.15.69:6829/1876101065","state":["exists","up"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6801","nonce":1806438563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6803","nonce":1806438563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6807","nonce":1806438563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6805","nonce":1806438563}]},"public_addr":"172.21.15.69:6801/1806438563","cluster_addr":"172.21.15.69:6803/1806438563","heartbeat_back_addr":"172.21.15.69:6807/1806438563","heartbeat_front_addr":"172.21.15.69:6805/1806438563","state":["exists","up"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6817","nonce":1593617983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6819","nonce":1593617983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6823","nonce":1593617983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6821","nonce":1593617983}]},"public_addr":"172.21.15.69:6817/1593617983","cluster_addr":"172.21.15.69:6819/1593617983","heartbeat_back_addr":"172.21.15.69:6823/1593617983","heartbeat_front_addr":"172.21.15.69:6821/1593617983","state":["exists","up"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6809","nonce":3950661294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6811","nonce":3950661294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6815","nonce":3950661294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6813","nonce":3950661294}]},"public_addr":"172.21.15.69:6809/3950661294","cluster_addr":"172.21.15.69:6811/3950661294","heartbeat_back_addr":"172.21.15.69:6815/3950661294","heartbeat_front_addr":"172.21.15.69:6813/3950661294","state":["exists","up"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6801","nonce":2508929872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6803","nonce":2508929872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6807","nonce":2508929872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6805","nonce":2508929872}]},"public_addr":"172.21.15.146:6801/2508929872","cluster_addr":"172.21.15.146:6803/2508929872","heartbeat_back_addr":"172.21.15.146:6807/2508929872","heartbeat_front_addr":"172.21.15.146:6805/2508929872","state":["exists","up"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6811","nonce":1310911138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6816","nonce":1310911138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6823","nonce":1310911138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6821","nonce":1310911138}]},"public_addr":"172.21.15.146:6811/1310911138","cluster_addr":"172.21.15.146:6816/1310911138","heartbeat_back_addr":"172.21.15.146:6823/1310911138","heartbeat_front_addr":"172.21.15.146:6821/1310911138","state":["exists","up"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6810","nonce":634688021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6813","nonce":634688021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6820","nonce":634688021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6817","nonce":634688021}]},"public_addr":"172.21.15.146:6810/634688021","cluster_addr":"172.21.15.146:6813/634688021","heartbeat_back_addr":"172.21.15.146:6820/634688021","heartbeat_front_addr":"172.21.15.146:6817/634688021","state":["exists","up"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6825","nonce":3814215935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6827","nonce":3814215935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6831","nonce":3814215935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6829","nonce":3814215935}]},"public_addr":"172.21.15.146:6825/3814215935","cluster_addr":"172.21.15.146:6827/3814215935","heartbeat_back_addr":"172.21.15.146:6831/3814215935","heartbeat_front_addr":"172.21.15.146:6829/3814215935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.820200+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.850189+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.764560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.866248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.807897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.869881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.782310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.827131+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:59:26.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.477+0000 7f08ab30d640 1 -- 172.21.15.69:0/766335835 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f08880744b0 msgr2=0x7f08880769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:26.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.477+0000 7f08ab30d640 1 --2- 172.21.15.69:0/766335835 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f08880744b0 0x7f08880769c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f08a40d23d0 tx=0x7f0894009040 comp rx=0 tx=0).stop 2024-04-25T10:59:26.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.477+0000 7f08ab30d640 1 -- 172.21.15.69:0/766335835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08a406d7e0 msgr2=0x7f08a409caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:26.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.477+0000 7f08ab30d640 1 --2- 172.21.15.69:0/766335835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08a406d7e0 0x7f08a409caa0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f08a000e870 tx=0x7f08a000ed30 comp rx=0 tx=0).stop 2024-04-25T10:59:26.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.477+0000 7f08ab30d640 1 -- 172.21.15.69:0/766335835 shutdown_connections 2024-04-25T10:59:26.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.477+0000 7f08ab30d640 1 --2- 172.21.15.69:0/766335835 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f08880744b0 0x7f08880769c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:26.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.477+0000 7f08ab30d640 1 --2- 172.21.15.69:0/766335835 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08a40cf610 0x7f08a40d1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:26.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.477+0000 7f08ab30d640 1 --2- 172.21.15.69:0/766335835 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08a40b7750 0x7f08a40d1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:26.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.477+0000 7f08ab30d640 1 --2- 172.21.15.69:0/766335835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08a406d7e0 0x7f08a409caa0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:26.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.477+0000 7f08ab30d640 1 -- 172.21.15.69:0/766335835 >> 172.21.15.69:0/766335835 conn(0x7f08a409b7d0 msgr2=0x7f08a4161040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:26.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.477+0000 7f08ab30d640 1 -- 172.21.15.69:0/766335835 shutdown_connections 2024-04-25T10:59:26.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.477+0000 7f08ab30d640 1 -- 172.21.15.69:0/766335835 wait complete. 2024-04-25T10:59:26.489 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:59:06.422674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:59:16.727980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:59:17.938486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:59:26.490 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:59:26.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.598+0000 7fca844d8640 1 Processor -- start 2024-04-25T10:59:26.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.598+0000 7fca844d8640 1 -- start start 2024-04-25T10:59:26.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.599+0000 7fca844d8640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fca7c164ad0 0x7fca7c0b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:26.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.599+0000 7fca844d8640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fca7c0b0a70 0x7fca7c0b0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:26.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.599+0000 7fca844d8640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca7c0d3170 con 0x7fca7c163f30 2024-04-25T10:59:26.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.599+0000 7fca844d8640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fca7c0d2f40 con 0x7fca7c0b0a70 2024-04-25T10:59:26.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.599+0000 7fca844d8640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fca7c164ee0 con 0x7fca7c164ad0 2024-04-25T10:59:26.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.599+0000 7fca82a75640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fca7c0b0a70 0x7fca7c0b0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:26.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.599+0000 7fca81a73640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fca7c163f30 0x7fca7c164340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:57164/0 (socket says 172.21.15.69:57164) 2024-04-25T10:59:26.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.599+0000 7fca82a75640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fca7c0b0a70 0x7fca7c0b0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56294/0 (socket says 172.21.15.69:56294) 2024-04-25T10:59:26.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.599+0000 7fca81a73640 1 -- 172.21.15.69:0/3527360650 learned_addr learned my addr 172.21.15.69:0/3527360650 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:26.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.600+0000 7fca82274640 1 --2- 172.21.15.69:0/3527360650 >> v2:172.21.15.146:3300/0 conn(0x7fca7c164ad0 0x7fca7c0b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:26.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.600+0000 7fca82a75640 1 -- 172.21.15.69:0/3527360650 >> v2:172.21.15.146:3300/0 conn(0x7fca7c164ad0 msgr2=0x7fca7c0b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:26.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.600+0000 7fca82a75640 1 --2- 172.21.15.69:0/3527360650 >> v2:172.21.15.146:3300/0 conn(0x7fca7c164ad0 0x7fca7c0b2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:26.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.600+0000 7fca82a75640 1 -- 172.21.15.69:0/3527360650 >> v1:172.21.15.69:6789/0 conn(0x7fca7c163f30 legacy=0x7fca7c164340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:26.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.600+0000 7fca82a75640 1 -- 172.21.15.69:0/3527360650 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca7c0d3450 con 0x7fca7c0b0a70 2024-04-25T10:59:26.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.600+0000 7fca81272640 1 -- 172.21.15.69:0/3527360650 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1264883817 0 0) 0x7fca7c0d3170 con 0x7fca7c163f30 2024-04-25T10:59:26.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.600+0000 7fca82a75640 1 --2- 172.21.15.69:0/3527360650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca7c0b0a70 0x7fca7c0b0e80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fca7c074530 tx=0x7fca7402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958daf73bbbf787b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:26.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.600+0000 7fca81272640 1 -- 172.21.15.69:0/3527360650 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74007740 con 0x7fca7c0b0a70 2024-04-25T10:59:26.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.600+0000 7fca81272640 1 -- 172.21.15.69:0/3527360650 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca740078e0 con 0x7fca7c0b0a70 2024-04-25T10:59:26.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.601+0000 7fca81272640 1 -- 172.21.15.69:0/3527360650 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74007cb0 con 0x7fca7c0b0a70 2024-04-25T10:59:26.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.601+0000 7fca844d8640 1 -- 172.21.15.69:0/3527360650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca7c0b0a70 msgr2=0x7fca7c0b0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:26.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.601+0000 7fca844d8640 1 --2- 172.21.15.69:0/3527360650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca7c0b0a70 0x7fca7c0b0e80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fca7c074530 tx=0x7fca7402a680 comp rx=0 tx=0).stop 2024-04-25T10:59:26.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.601+0000 7fca844d8640 1 -- 172.21.15.69:0/3527360650 shutdown_connections 2024-04-25T10:59:26.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.601+0000 7fca844d8640 1 --2- 172.21.15.69:0/3527360650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca7c0b0a70 0x7fca7c0b0e80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:26.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.601+0000 7fca844d8640 1 --2- 172.21.15.69:0/3527360650 >> v2:172.21.15.146:3300/0 conn(0x7fca7c164ad0 0x7fca7c0b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:26.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.601+0000 7fca844d8640 1 -- 172.21.15.69:0/3527360650 >> 172.21.15.69:0/3527360650 conn(0x7fca7c0d3aa0 msgr2=0x7fca7c0d3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:26.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.601+0000 7fca844d8640 1 -- 172.21.15.69:0/3527360650 shutdown_connections 2024-04-25T10:59:26.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.601+0000 7fca844d8640 1 -- 172.21.15.69:0/3527360650 wait complete. 2024-04-25T10:59:26.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.603+0000 7fca844d8640 1 Processor -- start 2024-04-25T10:59:26.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.603+0000 7fca844d8640 1 -- start start 2024-04-25T10:59:26.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.604+0000 7fca844d8640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fca7c0b0a70 0x7fca7c0b3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:26.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.604+0000 7fca844d8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca7c163f30 0x7fca7c0b5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:26.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.604+0000 7fca844d8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca7c164ad0 0x7fca7c0b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:26.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.604+0000 7fca844d8640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c06d9b0 con 0x7fca7c164ad0 2024-04-25T10:59:26.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.604+0000 7fca844d8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c0b31e0 con 0x7fca7c163f30 2024-04-25T10:59:26.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.604+0000 7fca844d8640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fca7c0b9430 con 0x7fca7c0b0a70 2024-04-25T10:59:26.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.604+0000 7fca82a75640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca7c164ad0 0x7fca7c0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:26.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.604+0000 7fca82a75640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca7c164ad0 0x7fca7c0b5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56308/0 (socket says 172.21.15.69:56308) 2024-04-25T10:59:26.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.604+0000 7fca82a75640 1 -- 172.21.15.69:0/4096754337 learned_addr learned my addr 172.21.15.69:0/4096754337 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:26.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.605+0000 7fca81a73640 1 --2- 172.21.15.69:0/4096754337 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca7c163f30 0x7fca7c0b5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:26.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.605+0000 7fca82274640 1 --2- 172.21.15.69:0/4096754337 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fca7c0b0a70 0x7fca7c0b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:26.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.605+0000 7fca82a75640 1 -- 172.21.15.69:0/4096754337 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fca7c0b0a70 msgr2=0x7fca7c0b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:26.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.605+0000 7fca82a75640 1 --2- 172.21.15.69:0/4096754337 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fca7c0b0a70 0x7fca7c0b3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:26.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.605+0000 7fca82a75640 1 -- 172.21.15.69:0/4096754337 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca7c163f30 msgr2=0x7fca7c0b5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:26.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.605+0000 7fca82a75640 1 --2- 172.21.15.69:0/4096754337 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca7c163f30 0x7fca7c0b5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:26.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.605+0000 7fca82a75640 1 -- 172.21.15.69:0/4096754337 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca7c0b3ea0 con 0x7fca7c164ad0 2024-04-25T10:59:26.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.605+0000 7fca82274640 1 --2- 172.21.15.69:0/4096754337 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fca7c0b0a70 0x7fca7c0b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:26.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.605+0000 7fca81a73640 1 --2- 172.21.15.69:0/4096754337 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca7c163f30 0x7fca7c0b5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:26.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.605+0000 7fca82a75640 1 --2- 172.21.15.69:0/4096754337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca7c164ad0 0x7fca7c0b5b70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fca74009e80 tx=0x7fca74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:26.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.605+0000 7fca72ffd640 1 -- 172.21.15.69:0/4096754337 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7403f070 con 0x7fca7c164ad0 2024-04-25T10:59:26.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.605+0000 7fca72ffd640 1 -- 172.21.15.69:0/4096754337 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca74031bc0 con 0x7fca7c164ad0 2024-04-25T10:59:26.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.605+0000 7fca844d8640 1 -- 172.21.15.69:0/4096754337 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca7c0cb6e0 con 0x7fca7c164ad0 2024-04-25T10:59:26.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.605+0000 7fca72ffd640 1 -- 172.21.15.69:0/4096754337 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74009450 con 0x7fca7c164ad0 2024-04-25T10:59:26.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.605+0000 7fca844d8640 1 -- 172.21.15.69:0/4096754337 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca7c0cbca0 con 0x7fca7c164ad0 2024-04-25T10:59:26.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.606+0000 7fca844d8640 1 -- 172.21.15.69:0/4096754337 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca7c1ecf70 con 0x7fca7c164ad0 2024-04-25T10:59:26.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.607+0000 7fca72ffd640 1 -- 172.21.15.69:0/4096754337 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fca74003990 con 0x7fca7c164ad0 2024-04-25T10:59:26.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.607+0000 7fca72ffd640 1 --2- 172.21.15.69:0/4096754337 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fca600744b0 0x7fca600769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:26.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.607+0000 7fca72ffd640 1 -- 172.21.15.69:0/4096754337 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fca740b6120 con 0x7fca7c164ad0 2024-04-25T10:59:26.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.607+0000 7fca82274640 1 --2- 172.21.15.69:0/4096754337 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fca600744b0 0x7fca600769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:26.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.608+0000 7fca82274640 1 --2- 172.21.15.69:0/4096754337 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fca600744b0 0x7fca600769c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fca78006fd0 tx=0x7fca78008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:26.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.612+0000 7fca72ffd640 1 -- 172.21.15.69:0/4096754337 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fca740808b0 con 0x7fca7c164ad0 2024-04-25T10:59:26.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.789+0000 7fca844d8640 1 -- 172.21.15.69:0/4096754337 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fca7c0731a0 con 0x7fca7c164ad0 2024-04-25T10:59:26.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.789+0000 7fca72ffd640 1 -- 172.21.15.69:0/4096754337 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fca740801b0 con 0x7fca7c164ad0 2024-04-25T10:59:26.790 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-25T10:59:26.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.791+0000 7fca844d8640 1 -- 172.21.15.69:0/4096754337 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fca600744b0 msgr2=0x7fca600769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:26.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.791+0000 7fca844d8640 1 --2- 172.21.15.69:0/4096754337 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fca600744b0 0x7fca600769c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fca78006fd0 tx=0x7fca78008040 comp rx=0 tx=0).stop 2024-04-25T10:59:26.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.791+0000 7fca844d8640 1 -- 172.21.15.69:0/4096754337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca7c164ad0 msgr2=0x7fca7c0b5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:26.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.791+0000 7fca844d8640 1 --2- 172.21.15.69:0/4096754337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca7c164ad0 0x7fca7c0b5b70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fca74009e80 tx=0x7fca74004a40 comp rx=0 tx=0).stop 2024-04-25T10:59:26.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.791+0000 7fca844d8640 1 -- 172.21.15.69:0/4096754337 shutdown_connections 2024-04-25T10:59:26.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.791+0000 7fca844d8640 1 --2- 172.21.15.69:0/4096754337 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fca600744b0 0x7fca600769c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:26.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.791+0000 7fca844d8640 1 --2- 172.21.15.69:0/4096754337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca7c164ad0 0x7fca7c0b5b70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:26.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.791+0000 7fca844d8640 1 --2- 172.21.15.69:0/4096754337 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca7c163f30 0x7fca7c0b5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:26.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.791+0000 7fca844d8640 1 --2- 172.21.15.69:0/4096754337 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fca7c0b0a70 0x7fca7c0b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:26.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.791+0000 7fca844d8640 1 -- 172.21.15.69:0/4096754337 >> 172.21.15.69:0/4096754337 conn(0x7fca7c0d3aa0 msgr2=0x7fca7c071fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:26.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.791+0000 7fca844d8640 1 -- 172.21.15.69:0/4096754337 shutdown_connections 2024-04-25T10:59:26.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.791+0000 7fca844d8640 1 -- 172.21.15.69:0/4096754337 wait complete. 2024-04-25T10:59:26.803 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:59:26.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.913+0000 7fa5eaa25640 1 Processor -- start 2024-04-25T10:59:26.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.913+0000 7fa5eaa25640 1 -- start start 2024-04-25T10:59:26.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.915+0000 7fa5eaa25640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5e40d1e20 0x7fa5e40d3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:26.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.915+0000 7fa5eaa25640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fa5e4164020 0x7fa5e4164430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:26.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.915+0000 7fa5eaa25640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5e4164970 con 0x7fa5e40b2e60 2024-04-25T10:59:26.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.915+0000 7fa5eaa25640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fa5e40d23f0 con 0x7fa5e4164020 2024-04-25T10:59:26.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.915+0000 7fa5eaa25640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa5e40d26d0 con 0x7fa5e40d1e20 2024-04-25T10:59:26.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.915+0000 7fa5e8fc2640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fa5e4164020 0x7fa5e4164430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:26.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.915+0000 7fa5e8fc2640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fa5e4164020 0x7fa5e4164430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56318/0 (socket says 172.21.15.69:56318) 2024-04-25T10:59:26.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.915+0000 7fa5e8fc2640 1 -- 172.21.15.69:0/3995707514 learned_addr learned my addr 172.21.15.69:0/3995707514 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:26.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.915+0000 7fa5e3fff640 1 --2- 172.21.15.69:0/3995707514 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5e40d1e20 0x7fa5e40d3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:26.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.915+0000 7fa5e8fc2640 1 -- 172.21.15.69:0/3995707514 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5e40d1e20 msgr2=0x7fa5e40d3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:26.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.915+0000 7fa5e8fc2640 1 --2- 172.21.15.69:0/3995707514 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5e40d1e20 0x7fa5e40d3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:26.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.915+0000 7fa5e8fc2640 1 -- 172.21.15.69:0/3995707514 >> v1:172.21.15.146:6789/0 conn(0x7fa5e40b2e60 legacy=0x7fa5e408d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:26.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.915+0000 7fa5e8fc2640 1 -- 172.21.15.69:0/3995707514 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e40d29b0 con 0x7fa5e4164020 2024-04-25T10:59:26.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.916+0000 7fa5e8fc2640 1 --2- 172.21.15.69:0/3995707514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5e4164020 0x7fa5e4164430 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc009e50 tx=0x7fa5dc02a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a631a39d7119f7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:26.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.916+0000 7fa5e2ffd640 1 -- 172.21.15.69:0/3995707514 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc036070 con 0x7fa5e4164020 2024-04-25T10:59:26.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.916+0000 7fa5e2ffd640 1 -- 172.21.15.69:0/3995707514 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5dc031600 con 0x7fa5e4164020 2024-04-25T10:59:26.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.916+0000 7fa5e2ffd640 1 -- 172.21.15.69:0/3995707514 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc0319d0 con 0x7fa5e4164020 2024-04-25T10:59:26.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.917+0000 7fa5eaa25640 1 -- 172.21.15.69:0/3995707514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5e4164020 msgr2=0x7fa5e4164430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:26.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.917+0000 7fa5eaa25640 1 --2- 172.21.15.69:0/3995707514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5e4164020 0x7fa5e4164430 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc009e50 tx=0x7fa5dc02a750 comp rx=0 tx=0).stop 2024-04-25T10:59:26.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.917+0000 7fa5eaa25640 1 -- 172.21.15.69:0/3995707514 shutdown_connections 2024-04-25T10:59:26.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.917+0000 7fa5eaa25640 1 --2- 172.21.15.69:0/3995707514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5e4164020 0x7fa5e4164430 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:26.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.917+0000 7fa5eaa25640 1 --2- 172.21.15.69:0/3995707514 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5e40d1e20 0x7fa5e40d3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:26.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.917+0000 7fa5eaa25640 1 -- 172.21.15.69:0/3995707514 >> 172.21.15.69:0/3995707514 conn(0x7fa5e40d0530 msgr2=0x7fa5e40d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:26.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.917+0000 7fa5eaa25640 1 -- 172.21.15.69:0/3995707514 shutdown_connections 2024-04-25T10:59:26.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.917+0000 7fa5eaa25640 1 -- 172.21.15.69:0/3995707514 wait complete. 2024-04-25T10:59:26.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.918+0000 7fa5eaa25640 1 Processor -- start 2024-04-25T10:59:26.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.919+0000 7fa5eaa25640 1 -- start start 2024-04-25T10:59:26.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.920+0000 7fa5eaa25640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5e40b2e60 0x7fa5e40cc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:26.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.920+0000 7fa5eaa25640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5e40d1e20 0x7fa5e40cc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:26.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.920+0000 7fa5eaa25640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5e4164020 0x7fa5e40ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:26.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.920+0000 7fa5eaa25640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e40ce070 con 0x7fa5e4164020 2024-04-25T10:59:26.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.920+0000 7fa5eaa25640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e40d2f70 con 0x7fa5e40b2e60 2024-04-25T10:59:26.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.920+0000 7fa5eaa25640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa5e40ce1f0 con 0x7fa5e40d1e20 2024-04-25T10:59:26.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.920+0000 7fa5e8fc2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5e4164020 0x7fa5e40ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:26.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.920+0000 7fa5e8fc2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5e4164020 0x7fa5e40ccb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56324/0 (socket says 172.21.15.69:56324) 2024-04-25T10:59:26.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.920+0000 7fa5e8fc2640 1 -- 172.21.15.69:0/3153027109 learned_addr learned my addr 172.21.15.69:0/3153027109 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:26.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.920+0000 7fa5e37fe640 1 --2- 172.21.15.69:0/3153027109 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5e40d1e20 0x7fa5e40cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:26.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.920+0000 7fa5e3fff640 1 --2- 172.21.15.69:0/3153027109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5e40b2e60 0x7fa5e40cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:26.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.920+0000 7fa5e8fc2640 1 -- 172.21.15.69:0/3153027109 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5e40d1e20 msgr2=0x7fa5e40cc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:26.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.920+0000 7fa5e8fc2640 1 --2- 172.21.15.69:0/3153027109 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5e40d1e20 0x7fa5e40cc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:26.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.920+0000 7fa5e8fc2640 1 -- 172.21.15.69:0/3153027109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5e40b2e60 msgr2=0x7fa5e40cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:26.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.920+0000 7fa5e8fc2640 1 --2- 172.21.15.69:0/3153027109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5e40b2e60 0x7fa5e40cc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:26.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.920+0000 7fa5e8fc2640 1 -- 172.21.15.69:0/3153027109 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e40cb950 con 0x7fa5e4164020 2024-04-25T10:59:26.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.920+0000 7fa5e3fff640 1 --2- 172.21.15.69:0/3153027109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5e40b2e60 0x7fa5e40cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:26.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.920+0000 7fa5e8fc2640 1 --2- 172.21.15.69:0/3153027109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5e4164020 0x7fa5e40ccb90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc009e50 tx=0x7fa5dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:26.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.921+0000 7fa5e0ff9640 1 -- 172.21.15.69:0/3153027109 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc036050 con 0x7fa5e4164020 2024-04-25T10:59:26.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.921+0000 7fa5e0ff9640 1 -- 172.21.15.69:0/3153027109 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5dc042070 con 0x7fa5e4164020 2024-04-25T10:59:26.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.921+0000 7fa5eaa25640 1 -- 172.21.15.69:0/3153027109 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5e40b7540 con 0x7fa5e4164020 2024-04-25T10:59:26.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.921+0000 7fa5e0ff9640 1 -- 172.21.15.69:0/3153027109 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc009510 con 0x7fa5e4164020 2024-04-25T10:59:26.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.921+0000 7fa5eaa25640 1 -- 172.21.15.69:0/3153027109 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5e40b7820 con 0x7fa5e4164020 2024-04-25T10:59:26.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.922+0000 7fa5eaa25640 1 -- 172.21.15.69:0/3153027109 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5e4050740 con 0x7fa5e4164020 2024-04-25T10:59:26.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.922+0000 7fa5e0ff9640 1 -- 172.21.15.69:0/3153027109 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa5dc007bd0 con 0x7fa5e4164020 2024-04-25T10:59:26.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.923+0000 7fa5e0ff9640 1 --2- 172.21.15.69:0/3153027109 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa5d00744e0 0x7fa5d00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:26.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.923+0000 7fa5e0ff9640 1 -- 172.21.15.69:0/3153027109 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa5dc0b60a0 con 0x7fa5e4164020 2024-04-25T10:59:26.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.923+0000 7fa5e3fff640 1 --2- 172.21.15.69:0/3153027109 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa5d00744e0 0x7fa5d00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:26.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.923+0000 7fa5e3fff640 1 --2- 172.21.15.69:0/3153027109 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa5d00744e0 0x7fa5d00769f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4006fd0 tx=0x7fa5d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:26.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:26.928+0000 7fa5e0ff9640 1 -- 172.21.15.69:0/3153027109 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa5e4164970 con 0x7fa5e4164020 2024-04-25T10:59:27.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.102+0000 7fa5eaa25640 1 -- 172.21.15.69:0/3153027109 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa5e4071000 con 0x7fa5e4164020 2024-04-25T10:59:27.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.103+0000 7fa5e0ff9640 1 -- 172.21.15.69:0/3153027109 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fa5dc080830 con 0x7fa5e4164020 2024-04-25T10:59:27.104 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-25T10:59:27.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.105+0000 7fa5eaa25640 1 -- 172.21.15.69:0/3153027109 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa5d00744e0 msgr2=0x7fa5d00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:27.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.105+0000 7fa5eaa25640 1 --2- 172.21.15.69:0/3153027109 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa5d00744e0 0x7fa5d00769f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4006fd0 tx=0x7fa5d4008040 comp rx=0 tx=0).stop 2024-04-25T10:59:27.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.105+0000 7fa5eaa25640 1 -- 172.21.15.69:0/3153027109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5e4164020 msgr2=0x7fa5e40ccb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:27.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.105+0000 7fa5eaa25640 1 --2- 172.21.15.69:0/3153027109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5e4164020 0x7fa5e40ccb90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc009e50 tx=0x7fa5dc004a40 comp rx=0 tx=0).stop 2024-04-25T10:59:27.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.105+0000 7fa5eaa25640 1 -- 172.21.15.69:0/3153027109 shutdown_connections 2024-04-25T10:59:27.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.105+0000 7fa5eaa25640 1 --2- 172.21.15.69:0/3153027109 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa5d00744e0 0x7fa5d00769f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:27.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.105+0000 7fa5eaa25640 1 --2- 172.21.15.69:0/3153027109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5e4164020 0x7fa5e40ccb90 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:27.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.105+0000 7fa5eaa25640 1 --2- 172.21.15.69:0/3153027109 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5e40d1e20 0x7fa5e40cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:27.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.105+0000 7fa5eaa25640 1 --2- 172.21.15.69:0/3153027109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5e40b2e60 0x7fa5e40cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:27.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.105+0000 7fa5eaa25640 1 -- 172.21.15.69:0/3153027109 >> 172.21.15.69:0/3153027109 conn(0x7fa5e40d0530 msgr2=0x7fa5e4159130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:27.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.105+0000 7fa5eaa25640 1 -- 172.21.15.69:0/3153027109 shutdown_connections 2024-04-25T10:59:27.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.105+0000 7fa5eaa25640 1 -- 172.21.15.69:0/3153027109 wait complete. 2024-04-25T10:59:27.117 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:59:27.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.225+0000 7f0f0e467640 1 Processor -- start 2024-04-25T10:59:27.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.225+0000 7f0f0e467640 1 -- start start 2024-04-25T10:59:27.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.227+0000 7f0f0e467640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0f080ce250 0x7f0f0815cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:27.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.227+0000 7f0f0e467640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0f0806d5d0 0x7f0f0806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:27.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.227+0000 7f0f0e467640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f0806df20 con 0x7f0f081458c0 2024-04-25T10:59:27.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.227+0000 7f0f0e467640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f0f080b4ec0 con 0x7f0f0806d5d0 2024-04-25T10:59:27.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.227+0000 7f0f0e467640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f0f08145cd0 con 0x7f0f080ce250 2024-04-25T10:59:27.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.227+0000 7f0f077fe640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f0f081458c0 0x7f0f080b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:57178/0 (socket says 172.21.15.69:57178) 2024-04-25T10:59:27.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.227+0000 7f0f077fe640 1 -- 172.21.15.69:0/1021277988 learned_addr learned my addr 172.21.15.69:0/1021277988 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:27.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.227+0000 7f0f0ca04640 1 --2- 172.21.15.69:0/1021277988 >> v2:172.21.15.69:3300/0 conn(0x7f0f0806d5d0 0x7f0f0806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:27.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.227+0000 7f0f06ffd640 1 -- 172.21.15.69:0/1021277988 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 301430089 0 0) 0x7f0efc001110 con 0x7f0f081458c0 2024-04-25T10:59:27.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.227+0000 7f0f07fff640 1 --2- 172.21.15.69:0/1021277988 >> v2:172.21.15.146:3300/0 conn(0x7f0f080ce250 0x7f0f0815cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:27.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.227+0000 7f0f06ffd640 1 -- 172.21.15.69:0/1021277988 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ef0003440 con 0x7f0f081458c0 2024-04-25T10:59:27.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.227+0000 7f0f0ca04640 1 -- 172.21.15.69:0/1021277988 >> v2:172.21.15.146:3300/0 conn(0x7f0f080ce250 msgr2=0x7f0f0815cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:27.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.227+0000 7f0f0ca04640 1 --2- 172.21.15.69:0/1021277988 >> v2:172.21.15.146:3300/0 conn(0x7f0f080ce250 0x7f0f0815cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:27.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.227+0000 7f0f0ca04640 1 -- 172.21.15.69:0/1021277988 >> v1:172.21.15.69:6789/0 conn(0x7f0f081458c0 legacy=0x7f0f080b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:27.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.227+0000 7f0f0ca04640 1 -- 172.21.15.69:0/1021277988 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f0815d5c0 con 0x7f0f0806d5d0 2024-04-25T10:59:27.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.228+0000 7f0f0ca04640 1 --2- 172.21.15.69:0/1021277988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f0806d5d0 0x7f0f0806d9e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0f080b4530 tx=0x7f0f0002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdbe7cae72e0b979 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:27.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.228+0000 7f0f06ffd640 1 -- 172.21.15.69:0/1021277988 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f00007740 con 0x7f0f0806d5d0 2024-04-25T10:59:27.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.228+0000 7f0f06ffd640 1 -- 172.21.15.69:0/1021277988 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f000078e0 con 0x7f0f0806d5d0 2024-04-25T10:59:27.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.228+0000 7f0f06ffd640 1 -- 172.21.15.69:0/1021277988 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f00007cb0 con 0x7f0f0806d5d0 2024-04-25T10:59:27.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.229+0000 7f0f0e467640 1 -- 172.21.15.69:0/1021277988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f0806d5d0 msgr2=0x7f0f0806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:27.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.229+0000 7f0f0e467640 1 --2- 172.21.15.69:0/1021277988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f0806d5d0 0x7f0f0806d9e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0f080b4530 tx=0x7f0f0002a680 comp rx=0 tx=0).stop 2024-04-25T10:59:27.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.229+0000 7f0f0e467640 1 -- 172.21.15.69:0/1021277988 shutdown_connections 2024-04-25T10:59:27.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.229+0000 7f0f0e467640 1 --2- 172.21.15.69:0/1021277988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f0806d5d0 0x7f0f0806d9e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:27.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.229+0000 7f0f0e467640 1 --2- 172.21.15.69:0/1021277988 >> v2:172.21.15.146:3300/0 conn(0x7f0f080ce250 0x7f0f0815cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:27.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.229+0000 7f0f0e467640 1 -- 172.21.15.69:0/1021277988 >> 172.21.15.69:0/1021277988 conn(0x7f0f08073090 msgr2=0x7f0f080734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:27.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.229+0000 7f0f0e467640 1 -- 172.21.15.69:0/1021277988 shutdown_connections 2024-04-25T10:59:27.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.229+0000 7f0f0e467640 1 -- 172.21.15.69:0/1021277988 wait complete. 2024-04-25T10:59:27.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.231+0000 7f0f0e467640 1 Processor -- start 2024-04-25T10:59:27.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.231+0000 7f0f0e467640 1 -- start start 2024-04-25T10:59:27.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.232+0000 7f0f0e467640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f0806d5d0 0x7f0f0813cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:27.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.232+0000 7f0f0e467640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f080ce250 0x7f0f081f6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:27.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.232+0000 7f0f0e467640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f081458c0 0x7f0f080b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:27.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.232+0000 7f0f0e467640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0806f640 con 0x7f0f0806d5d0 2024-04-25T10:59:27.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.232+0000 7f0f0e467640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0815db80 con 0x7f0f080ce250 2024-04-25T10:59:27.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.232+0000 7f0f0e467640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0f0815de60 con 0x7f0f081458c0 2024-04-25T10:59:27.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.232+0000 7f0f07fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f0806d5d0 0x7f0f0813cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:27.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.232+0000 7f0f07fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f0806d5d0 0x7f0f0813cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56350/0 (socket says 172.21.15.69:56350) 2024-04-25T10:59:27.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.232+0000 7f0f07fff640 1 -- 172.21.15.69:0/1850651450 learned_addr learned my addr 172.21.15.69:0/1850651450 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:27.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.232+0000 7f0f0ca04640 1 --2- 172.21.15.69:0/1850651450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f081458c0 0x7f0f080b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:27.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.232+0000 7f0f077fe640 1 --2- 172.21.15.69:0/1850651450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f080ce250 0x7f0f081f6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:27.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.232+0000 7f0f07fff640 1 -- 172.21.15.69:0/1850651450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f081458c0 msgr2=0x7f0f080b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:27.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.232+0000 7f0f07fff640 1 --2- 172.21.15.69:0/1850651450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f081458c0 0x7f0f080b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:27.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.232+0000 7f0f07fff640 1 -- 172.21.15.69:0/1850651450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f080ce250 msgr2=0x7f0f081f6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:27.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.232+0000 7f0f07fff640 1 --2- 172.21.15.69:0/1850651450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f080ce250 0x7f0f081f6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:27.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.232+0000 7f0f07fff640 1 -- 172.21.15.69:0/1850651450 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f081525a0 con 0x7f0f0806d5d0 2024-04-25T10:59:27.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.233+0000 7f0f07fff640 1 --2- 172.21.15.69:0/1850651450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f0806d5d0 0x7f0f0813cc30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800d5d0 tx=0x7f0ef800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:27.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.233+0000 7f0f04ff9640 1 -- 172.21.15.69:0/1850651450 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8004240 con 0x7f0f0806d5d0 2024-04-25T10:59:27.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.233+0000 7f0f04ff9640 1 -- 172.21.15.69:0/1850651450 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ef80043e0 con 0x7f0f0806d5d0 2024-04-25T10:59:27.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.233+0000 7f0f04ff9640 1 -- 172.21.15.69:0/1850651450 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8003830 con 0x7f0f0806d5d0 2024-04-25T10:59:27.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.233+0000 7f0f0e467640 1 -- 172.21.15.69:0/1850651450 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f08152880 con 0x7f0f0806d5d0 2024-04-25T10:59:27.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.233+0000 7f0f0e467640 1 -- 172.21.15.69:0/1850651450 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f08152e40 con 0x7f0f0806d5d0 2024-04-25T10:59:27.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.234+0000 7f0f0e467640 1 -- 172.21.15.69:0/1850651450 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f08050740 con 0x7f0f0806d5d0 2024-04-25T10:59:27.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.235+0000 7f0f04ff9640 1 -- 172.21.15.69:0/1850651450 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0ef80039d0 con 0x7f0f0806d5d0 2024-04-25T10:59:27.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.235+0000 7f0f04ff9640 1 --2- 172.21.15.69:0/1850651450 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0ef40744b0 0x7f0ef40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:27.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.235+0000 7f0f077fe640 1 --2- 172.21.15.69:0/1850651450 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0ef40744b0 0x7f0ef40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:27.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.235+0000 7f0f04ff9640 1 -- 172.21.15.69:0/1850651450 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0ef8095360 con 0x7f0f0806d5d0 2024-04-25T10:59:27.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.235+0000 7f0f077fe640 1 --2- 172.21.15.69:0/1850651450 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0ef40744b0 0x7f0ef40769c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0f080b1270 tx=0x7f0efc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:27.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.240+0000 7f0f04ff9640 1 -- 172.21.15.69:0/1850651450 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0ef805f4a0 con 0x7f0f0806d5d0 2024-04-25T10:59:27.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.415+0000 7f0f0e467640 1 -- 172.21.15.69:0/1850651450 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0f080ceec0 con 0x7f0f0806d5d0 2024-04-25T10:59:27.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.415+0000 7f0f04ff9640 1 -- 172.21.15.69:0/1850651450 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f0ef80637a0 con 0x7f0f0806d5d0 2024-04-25T10:59:27.416 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-25T10:59:27.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.417+0000 7f0f0e467640 1 -- 172.21.15.69:0/1850651450 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0ef40744b0 msgr2=0x7f0ef40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:27.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.417+0000 7f0f0e467640 1 --2- 172.21.15.69:0/1850651450 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0ef40744b0 0x7f0ef40769c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0f080b1270 tx=0x7f0efc009040 comp rx=0 tx=0).stop 2024-04-25T10:59:27.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.417+0000 7f0f0e467640 1 -- 172.21.15.69:0/1850651450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f0806d5d0 msgr2=0x7f0f0813cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:27.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.417+0000 7f0f0e467640 1 --2- 172.21.15.69:0/1850651450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f0806d5d0 0x7f0f0813cc30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800d5d0 tx=0x7f0ef800da90 comp rx=0 tx=0).stop 2024-04-25T10:59:27.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.417+0000 7f0f0e467640 1 -- 172.21.15.69:0/1850651450 shutdown_connections 2024-04-25T10:59:27.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.417+0000 7f0f0e467640 1 --2- 172.21.15.69:0/1850651450 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0ef40744b0 0x7f0ef40769c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:27.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.417+0000 7f0f0e467640 1 --2- 172.21.15.69:0/1850651450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f081458c0 0x7f0f080b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:27.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.417+0000 7f0f0e467640 1 --2- 172.21.15.69:0/1850651450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f080ce250 0x7f0f081f6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:27.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.417+0000 7f0f0e467640 1 --2- 172.21.15.69:0/1850651450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f0806d5d0 0x7f0f0813cc30 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:27.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.417+0000 7f0f0e467640 1 -- 172.21.15.69:0/1850651450 >> 172.21.15.69:0/1850651450 conn(0x7f0f08073090 msgr2=0x7f0f080b5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:27.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.417+0000 7f0f0e467640 1 -- 172.21.15.69:0/1850651450 shutdown_connections 2024-04-25T10:59:27.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.417+0000 7f0f0e467640 1 -- 172.21.15.69:0/1850651450 wait complete. 2024-04-25T10:59:27.429 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:59:27.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.536+0000 7f66acf72640 1 Processor -- start 2024-04-25T10:59:27.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.536+0000 7f66acf72640 1 -- start start 2024-04-25T10:59:27.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.537+0000 7f66acf72640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f66a80b91d0 0x7f66a8161e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:27.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.537+0000 7f66acf72640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66a81623c0 0x7f66a81627d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:27.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.537+0000 7f66acf72640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66a80cfc30 con 0x7f66a80ce760 2024-04-25T10:59:27.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.537+0000 7f66acf72640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f66a80d1ff0 con 0x7f66a80b91d0 2024-04-25T10:59:27.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.537+0000 7f66acf72640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f66a80d2890 con 0x7f66a81623c0 2024-04-25T10:59:27.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.537+0000 7f66a659c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f66a80b91d0 0x7f66a8161e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:27.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.537+0000 7f66a6d9d640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f66a80ce760 0x7f66a80ceb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:57190/0 (socket says 172.21.15.69:57190) 2024-04-25T10:59:27.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.537+0000 7f66a659c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f66a80b91d0 0x7f66a8161e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56360/0 (socket says 172.21.15.69:56360) 2024-04-25T10:59:27.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.537+0000 7f66a6d9d640 1 -- 172.21.15.69:0/2836110772 learned_addr learned my addr 172.21.15.69:0/2836110772 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:27.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.538+0000 7f669ffff640 1 --2- 172.21.15.69:0/2836110772 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66a81623c0 0x7f66a81627d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:27.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.538+0000 7f66a659c640 1 -- 172.21.15.69:0/2836110772 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66a81623c0 msgr2=0x7f66a81627d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:27.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.538+0000 7f66a659c640 1 --2- 172.21.15.69:0/2836110772 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66a81623c0 0x7f66a81627d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:27.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.538+0000 7f66a659c640 1 -- 172.21.15.69:0/2836110772 >> v1:172.21.15.69:6789/0 conn(0x7f66a80ce760 legacy=0x7f66a80ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:27.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.538+0000 7f66a659c640 1 -- 172.21.15.69:0/2836110772 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a80d2240 con 0x7f66a80b91d0 2024-04-25T10:59:27.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.538+0000 7f66a659c640 1 --2- 172.21.15.69:0/2836110772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66a80b91d0 0x7f66a8161e80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f66980097a0 tx=0x7f669802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c605af4f265d7572 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:27.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.538+0000 7f66a5d9b640 1 -- 172.21.15.69:0/2836110772 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698037070 con 0x7f66a80b91d0 2024-04-25T10:59:27.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.539+0000 7f66a5d9b640 1 -- 172.21.15.69:0/2836110772 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6698032440 con 0x7f66a80b91d0 2024-04-25T10:59:27.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.539+0000 7f66a5d9b640 1 -- 172.21.15.69:0/2836110772 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698032810 con 0x7f66a80b91d0 2024-04-25T10:59:27.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.539+0000 7f66acf72640 1 -- 172.21.15.69:0/2836110772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66a80b91d0 msgr2=0x7f66a8161e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:27.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.539+0000 7f66acf72640 1 --2- 172.21.15.69:0/2836110772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66a80b91d0 0x7f66a8161e80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f66980097a0 tx=0x7f669802a180 comp rx=0 tx=0).stop 2024-04-25T10:59:27.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.539+0000 7f66acf72640 1 -- 172.21.15.69:0/2836110772 shutdown_connections 2024-04-25T10:59:27.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.539+0000 7f66acf72640 1 --2- 172.21.15.69:0/2836110772 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66a81623c0 0x7f66a81627d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:27.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.539+0000 7f66acf72640 1 --2- 172.21.15.69:0/2836110772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66a80b91d0 0x7f66a8161e80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:27.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.539+0000 7f66acf72640 1 -- 172.21.15.69:0/2836110772 >> 172.21.15.69:0/2836110772 conn(0x7f66a80d3aa0 msgr2=0x7f66a80d3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:27.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.539+0000 7f66acf72640 1 -- 172.21.15.69:0/2836110772 shutdown_connections 2024-04-25T10:59:27.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.539+0000 7f66acf72640 1 -- 172.21.15.69:0/2836110772 wait complete. 2024-04-25T10:59:27.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.541+0000 7f66acf72640 1 Processor -- start 2024-04-25T10:59:27.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.541+0000 7f66acf72640 1 -- start start 2024-04-25T10:59:27.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.542+0000 7f66acf72640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66a80b91d0 0x7f66a80ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:27.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.542+0000 7f66acf72640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66a80ce760 0x7f66a815ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:27.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.542+0000 7f66acf72640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66a81623c0 0x7f66a80b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:27.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.542+0000 7f66acf72640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f66a80b2320 con 0x7f66a81623c0 2024-04-25T10:59:27.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.542+0000 7f66acf72640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f66a8162d10 con 0x7f66a80b91d0 2024-04-25T10:59:27.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.542+0000 7f66acf72640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f66a80b24a0 con 0x7f66a80ce760 2024-04-25T10:59:27.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.543+0000 7f66a6d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66a81623c0 0x7f66a80b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:27.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.543+0000 7f66a6d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66a81623c0 0x7f66a80b43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56362/0 (socket says 172.21.15.69:56362) 2024-04-25T10:59:27.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.543+0000 7f66a6d9d640 1 -- 172.21.15.69:0/791620815 learned_addr learned my addr 172.21.15.69:0/791620815 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:27.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.543+0000 7f669ffff640 1 --2- 172.21.15.69:0/791620815 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66a80ce760 0x7f66a815ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:27.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.543+0000 7f66a659c640 1 --2- 172.21.15.69:0/791620815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66a80b91d0 0x7f66a80ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:27.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.543+0000 7f66a6d9d640 1 -- 172.21.15.69:0/791620815 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66a80ce760 msgr2=0x7f66a815ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:27.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.543+0000 7f66a6d9d640 1 --2- 172.21.15.69:0/791620815 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66a80ce760 0x7f66a815ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:27.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.543+0000 7f66a6d9d640 1 -- 172.21.15.69:0/791620815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66a80b91d0 msgr2=0x7f66a80ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:27.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.543+0000 7f66a6d9d640 1 --2- 172.21.15.69:0/791620815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66a80b91d0 0x7f66a80ca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:27.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.543+0000 7f66a6d9d640 1 -- 172.21.15.69:0/791620815 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a80c8970 con 0x7f66a81623c0 2024-04-25T10:59:27.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.543+0000 7f66a6d9d640 1 --2- 172.21.15.69:0/791620815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66a81623c0 0x7f66a80b43c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f66a000cab0 tx=0x7f66a000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:27.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.543+0000 7f669effd640 1 -- 172.21.15.69:0/791620815 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0017070 con 0x7f66a81623c0 2024-04-25T10:59:27.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.543+0000 7f669effd640 1 -- 172.21.15.69:0/791620815 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66a000ac30 con 0x7f66a81623c0 2024-04-25T10:59:27.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.543+0000 7f66acf72640 1 -- 172.21.15.69:0/791620815 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a80c8c50 con 0x7f66a81623c0 2024-04-25T10:59:27.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.543+0000 7f669effd640 1 -- 172.21.15.69:0/791620815 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0012400 con 0x7f66a81623c0 2024-04-25T10:59:27.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.543+0000 7f66acf72640 1 -- 172.21.15.69:0/791620815 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a80c9210 con 0x7f66a81623c0 2024-04-25T10:59:27.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.544+0000 7f66acf72640 1 -- 172.21.15.69:0/791620815 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66a8050740 con 0x7f66a81623c0 2024-04-25T10:59:27.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.545+0000 7f669effd640 1 -- 172.21.15.69:0/791620815 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f66a001a070 con 0x7f66a81623c0 2024-04-25T10:59:27.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.545+0000 7f669effd640 1 --2- 172.21.15.69:0/791620815 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f66880744b0 0x7f66880769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:27.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.546+0000 7f66a659c640 1 --2- 172.21.15.69:0/791620815 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f66880744b0 0x7f66880769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:27.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.546+0000 7f669effd640 1 -- 172.21.15.69:0/791620815 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f66a0094d90 con 0x7f66a81623c0 2024-04-25T10:59:27.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.546+0000 7f66a659c640 1 --2- 172.21.15.69:0/791620815 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f66880744b0 0x7f66880769c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6698034000 tx=0x7f669802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:27.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.550+0000 7f669effd640 1 -- 172.21.15.69:0/791620815 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f66a005ef60 con 0x7f66a81623c0 2024-04-25T10:59:27.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.768+0000 7f66acf72640 1 -- 172.21.15.69:0/791620815 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66a80b6a70 con 0x7f66a81623c0 2024-04-25T10:59:27.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.768+0000 7f669effd640 1 -- 172.21.15.69:0/791620815 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5178 (secure 0 0 0) 0x7f66a0063260 con 0x7f66a81623c0 2024-04-25T10:59:27.769 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:27.769 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/2361834023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2361834023},{"type":"v1","addr":"172.21.15.146:6833","nonce":2361834023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1848276963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1848276963},{"type":"v1","addr":"172.21.15.69:6837","nonce":1848276963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1826742846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1826742846},{"type":"v1","addr":"172.21.15.69:6836","nonce":1826742846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:59:18.976232+0000","modified":"2024-04-25T10:59:20.983284+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6835/908008797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":908008797},{"type":"v1","addr":"172.21.15.146:6835","nonce":908008797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-04-25T10:59:27.769 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 6 2024-04-25T10:59:27.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.770+0000 7f66acf72640 1 -- 172.21.15.69:0/791620815 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f66880744b0 msgr2=0x7f66880769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:27.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.770+0000 7f66acf72640 1 --2- 172.21.15.69:0/791620815 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f66880744b0 0x7f66880769c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6698034000 tx=0x7f669802e000 comp rx=0 tx=0).stop 2024-04-25T10:59:27.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.770+0000 7f66acf72640 1 -- 172.21.15.69:0/791620815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66a81623c0 msgr2=0x7f66a80b43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:27.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.770+0000 7f66acf72640 1 --2- 172.21.15.69:0/791620815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66a81623c0 0x7f66a80b43c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f66a000cab0 tx=0x7f66a000cf70 comp rx=0 tx=0).stop 2024-04-25T10:59:27.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.770+0000 7f66acf72640 1 -- 172.21.15.69:0/791620815 shutdown_connections 2024-04-25T10:59:27.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.770+0000 7f66acf72640 1 --2- 172.21.15.69:0/791620815 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f66880744b0 0x7f66880769c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:27.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.770+0000 7f66acf72640 1 --2- 172.21.15.69:0/791620815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66a81623c0 0x7f66a80b43c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:27.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.770+0000 7f66acf72640 1 --2- 172.21.15.69:0/791620815 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66a80ce760 0x7f66a815ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:27.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.770+0000 7f66acf72640 1 --2- 172.21.15.69:0/791620815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66a80b91d0 0x7f66a80ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:27.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.770+0000 7f66acf72640 1 -- 172.21.15.69:0/791620815 >> 172.21.15.69:0/791620815 conn(0x7f66a80d3aa0 msgr2=0x7f66a80684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:27.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.770+0000 7f66acf72640 1 -- 172.21.15.69:0/791620815 shutdown_connections 2024-04-25T10:59:27.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.770+0000 7f66acf72640 1 -- 172.21.15.69:0/791620815 wait complete. 2024-04-25T10:59:27.783 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:59:27.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.889+0000 7fbe24abf640 1 Processor -- start 2024-04-25T10:59:27.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.889+0000 7fbe24abf640 1 -- start start 2024-04-25T10:59:27.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.891+0000 7fbe24abf640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fbe2006ccd0 0x7fbe200d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:27.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.891+0000 7fbe24abf640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbe200d4a50 0x7fbe200d4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:27.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.891+0000 7fbe24abf640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe200d1850 con 0x7fbe200d0e00 2024-04-25T10:59:27.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.891+0000 7fbe24abf640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fbe20160de0 con 0x7fbe2006ccd0 2024-04-25T10:59:27.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.891+0000 7fbe24abf640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fbe201610c0 con 0x7fbe200d4a50 2024-04-25T10:59:27.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.891+0000 7fbe1dd9b640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fbe2006ccd0 0x7fbe200d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:27.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.891+0000 7fbe1dd9b640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fbe2006ccd0 0x7fbe200d1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56370/0 (socket says 172.21.15.69:56370) 2024-04-25T10:59:27.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.891+0000 7fbe1e59c640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fbe200d0e00 0x7fbe2006c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:57202/0 (socket says 172.21.15.69:57202) 2024-04-25T10:59:27.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.891+0000 7fbe1dd9b640 1 -- 172.21.15.69:0/3566183209 learned_addr learned my addr 172.21.15.69:0/3566183209 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:27.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.891+0000 7fbe1ed9d640 1 --2- 172.21.15.69:0/3566183209 >> v2:172.21.15.146:3300/0 conn(0x7fbe200d4a50 0x7fbe200d4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:27.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.891+0000 7fbe1dd9b640 1 -- 172.21.15.69:0/3566183209 >> v2:172.21.15.146:3300/0 conn(0x7fbe200d4a50 msgr2=0x7fbe200d4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:27.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.891+0000 7fbe1dd9b640 1 --2- 172.21.15.69:0/3566183209 >> v2:172.21.15.146:3300/0 conn(0x7fbe200d4a50 0x7fbe200d4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:27.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.891+0000 7fbe1dd9b640 1 -- 172.21.15.69:0/3566183209 >> v1:172.21.15.69:6789/0 conn(0x7fbe200d0e00 legacy=0x7fbe2006c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:27.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.891+0000 7fbe1dd9b640 1 -- 172.21.15.69:0/3566183209 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe201613a0 con 0x7fbe2006ccd0 2024-04-25T10:59:27.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.891+0000 7fbe1d59a640 1 -- 172.21.15.69:0/3566183209 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4116611152 0 0) 0x7fbe200d1850 con 0x7fbe200d0e00 2024-04-25T10:59:27.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.891+0000 7fbe1ed9d640 1 --2- 172.21.15.69:0/3566183209 >> v2:172.21.15.146:3300/0 conn(0x7fbe200d4a50 0x7fbe200d4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:27.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.892+0000 7fbe1dd9b640 1 --2- 172.21.15.69:0/3566183209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe2006ccd0 0x7fbe200d1210 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbe200d1750 tx=0x7fbe1402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f420bf0d54e923 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:27.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.892+0000 7fbe1d59a640 1 -- 172.21.15.69:0/3566183209 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe14037070 con 0x7fbe2006ccd0 2024-04-25T10:59:27.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.892+0000 7fbe1d59a640 1 -- 172.21.15.69:0/3566183209 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe14032440 con 0x7fbe2006ccd0 2024-04-25T10:59:27.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.892+0000 7fbe1d59a640 1 -- 172.21.15.69:0/3566183209 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe14032810 con 0x7fbe2006ccd0 2024-04-25T10:59:27.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.892+0000 7fbe24abf640 1 -- 172.21.15.69:0/3566183209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe2006ccd0 msgr2=0x7fbe200d1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:27.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.893+0000 7fbe24abf640 1 --2- 172.21.15.69:0/3566183209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe2006ccd0 0x7fbe200d1210 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbe200d1750 tx=0x7fbe1402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:59:27.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.893+0000 7fbe24abf640 1 -- 172.21.15.69:0/3566183209 shutdown_connections 2024-04-25T10:59:27.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.893+0000 7fbe24abf640 1 --2- 172.21.15.69:0/3566183209 >> v2:172.21.15.146:3300/0 conn(0x7fbe200d4a50 0x7fbe200d4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:27.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.893+0000 7fbe24abf640 1 --2- 172.21.15.69:0/3566183209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe2006ccd0 0x7fbe200d1210 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:27.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.893+0000 7fbe24abf640 1 -- 172.21.15.69:0/3566183209 >> 172.21.15.69:0/3566183209 conn(0x7fbe200d5d90 msgr2=0x7fbe200d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:27.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.893+0000 7fbe24abf640 1 -- 172.21.15.69:0/3566183209 shutdown_connections 2024-04-25T10:59:27.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.893+0000 7fbe24abf640 1 -- 172.21.15.69:0/3566183209 wait complete. 2024-04-25T10:59:27.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.895+0000 7fbe24abf640 1 Processor -- start 2024-04-25T10:59:27.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.895+0000 7fbe24abf640 1 -- start start 2024-04-25T10:59:27.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.896+0000 7fbe24abf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe200d0e00 0x7fbe200ca020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:27.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.896+0000 7fbe24abf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe200d4a50 0x7fbe200cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:27.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.896+0000 7fbe24abf640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbe200c97f0 0x7fbe200cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:27.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.896+0000 7fbe24abf640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbe20164e00 con 0x7fbe200d0e00 2024-04-25T10:59:27.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.896+0000 7fbe24abf640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbe200b5320 con 0x7fbe200d4a50 2024-04-25T10:59:27.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.896+0000 7fbe24abf640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbe20164f80 con 0x7fbe200c97f0 2024-04-25T10:59:27.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.896+0000 7fbe1e59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe200d0e00 0x7fbe200ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:27.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.896+0000 7fbe1e59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe200d0e00 0x7fbe200ca020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56372/0 (socket says 172.21.15.69:56372) 2024-04-25T10:59:27.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.896+0000 7fbe1e59c640 1 -- 172.21.15.69:0/2907102875 learned_addr learned my addr 172.21.15.69:0/2907102875 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:27.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.896+0000 7fbe1ed9d640 1 --2- 172.21.15.69:0/2907102875 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbe200c97f0 0x7fbe200cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:27.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.896+0000 7fbe1dd9b640 1 --2- 172.21.15.69:0/2907102875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe200d4a50 0x7fbe200cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:27.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.896+0000 7fbe1e59c640 1 -- 172.21.15.69:0/2907102875 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbe200c97f0 msgr2=0x7fbe200cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:27.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.896+0000 7fbe1e59c640 1 --2- 172.21.15.69:0/2907102875 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbe200c97f0 0x7fbe200cc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:27.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.896+0000 7fbe1e59c640 1 -- 172.21.15.69:0/2907102875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe200d4a50 msgr2=0x7fbe200cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:27.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.896+0000 7fbe1e59c640 1 --2- 172.21.15.69:0/2907102875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe200d4a50 0x7fbe200cbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:27.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.896+0000 7fbe1e59c640 1 -- 172.21.15.69:0/2907102875 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe200ce830 con 0x7fbe200d0e00 2024-04-25T10:59:27.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.896+0000 7fbe1dd9b640 1 --2- 172.21.15.69:0/2907102875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe200d4a50 0x7fbe200cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:27.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.896+0000 7fbe1e59c640 1 --2- 172.21.15.69:0/2907102875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe200d0e00 0x7fbe200ca020 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbe1000c430 tx=0x7fbe1000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:27.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.897+0000 7fbe0effd640 1 -- 172.21.15.69:0/2907102875 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe100074e0 con 0x7fbe200d0e00 2024-04-25T10:59:27.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.897+0000 7fbe0effd640 1 -- 172.21.15.69:0/2907102875 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe100047e0 con 0x7fbe200d0e00 2024-04-25T10:59:27.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.897+0000 7fbe24abf640 1 -- 172.21.15.69:0/2907102875 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe200ceb10 con 0x7fbe200d0e00 2024-04-25T10:59:27.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.897+0000 7fbe0effd640 1 -- 172.21.15.69:0/2907102875 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe10004bd0 con 0x7fbe200d0e00 2024-04-25T10:59:27.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.897+0000 7fbe24abf640 1 -- 172.21.15.69:0/2907102875 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe200cf0d0 con 0x7fbe200d0e00 2024-04-25T10:59:27.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.898+0000 7fbe24abf640 1 -- 172.21.15.69:0/2907102875 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe20050740 con 0x7fbe200d0e00 2024-04-25T10:59:27.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.898+0000 7fbe0effd640 1 -- 172.21.15.69:0/2907102875 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbe10011070 con 0x7fbe200d0e00 2024-04-25T10:59:27.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.899+0000 7fbe0effd640 1 --2- 172.21.15.69:0/2907102875 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fbe000744b0 0x7fbe000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:27.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.899+0000 7fbe1dd9b640 1 --2- 172.21.15.69:0/2907102875 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fbe000744b0 0x7fbe000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:27.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.899+0000 7fbe0effd640 1 -- 172.21.15.69:0/2907102875 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbe10094ed0 con 0x7fbe200d0e00 2024-04-25T10:59:27.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.899+0000 7fbe1dd9b640 1 --2- 172.21.15.69:0/2907102875 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fbe000744b0 0x7fbe000769c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbe14034000 tx=0x7fbe1402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:27.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:27.903+0000 7fbe0effd640 1 -- 172.21.15.69:0/2907102875 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbe1005f660 con 0x7fbe200d0e00 2024-04-25T10:59:28.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.077+0000 7fbe24abf640 1 -- 172.21.15.69:0/2907102875 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbe200d4320 con 0x7fbe200d0e00 2024-04-25T10:59:28.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.078+0000 7fbe0effd640 1 -- 172.21.15.69:0/2907102875 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7fbe1005ef60 con 0x7fbe200d0e00 2024-04-25T10:59:28.079 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:28.079 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":20,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T10:59:20.630151+0000","last_up_change":"2024-04-25T10:59:05.546839+0000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:59:06.422674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:59:16.727980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:59:17.938486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6825","nonce":1876101065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6827","nonce":1876101065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6831","nonce":1876101065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6829","nonce":1876101065}]},"public_addr":"172.21.15.69:6825/1876101065","cluster_addr":"172.21.15.69:6827/1876101065","heartbeat_back_addr":"172.21.15.69:6831/1876101065","heartbeat_front_addr":"172.21.15.69:6829/1876101065","state":["exists","up"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6801","nonce":1806438563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6803","nonce":1806438563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6807","nonce":1806438563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6805","nonce":1806438563}]},"public_addr":"172.21.15.69:6801/1806438563","cluster_addr":"172.21.15.69:6803/1806438563","heartbeat_back_addr":"172.21.15.69:6807/1806438563","heartbeat_front_addr":"172.21.15.69:6805/1806438563","state":["exists","up"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6817","nonce":1593617983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6819","nonce":1593617983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6823","nonce":1593617983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6821","nonce":1593617983}]},"public_addr":"172.21.15.69:6817/1593617983","cluster_addr":"172.21.15.69:6819/1593617983","heartbeat_back_addr":"172.21.15.69:6823/1593617983","heartbeat_front_addr":"172.21.15.69:6821/1593617983","state":["exists","up"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6809","nonce":3950661294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6811","nonce":3950661294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6815","nonce":3950661294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6813","nonce":3950661294}]},"public_addr":"172.21.15.69:6809/3950661294","cluster_addr":"172.21.15.69:6811/3950661294","heartbeat_back_addr":"172.21.15.69:6815/3950661294","heartbeat_front_addr":"172.21.15.69:6813/3950661294","state":["exists","up"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6801","nonce":2508929872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6803","nonce":2508929872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6807","nonce":2508929872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6805","nonce":2508929872}]},"public_addr":"172.21.15.146:6801/2508929872","cluster_addr":"172.21.15.146:6803/2508929872","heartbeat_back_addr":"172.21.15.146:6807/2508929872","heartbeat_front_addr":"172.21.15.146:6805/2508929872","state":["exists","up"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6811","nonce":1310911138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6816","nonce":1310911138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6823","nonce":1310911138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6821","nonce":1310911138}]},"public_addr":"172.21.15.146:6811/1310911138","cluster_addr":"172.21.15.146:6816/1310911138","heartbeat_back_addr":"172.21.15.146:6823/1310911138","heartbeat_front_addr":"172.21.15.146:6821/1310911138","state":["exists","up"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6810","nonce":634688021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6813","nonce":634688021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6820","nonce":634688021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6817","nonce":634688021}]},"public_addr":"172.21.15.146:6810/634688021","cluster_addr":"172.21.15.146:6813/634688021","heartbeat_back_addr":"172.21.15.146:6820/634688021","heartbeat_front_addr":"172.21.15.146:6817/634688021","state":["exists","up"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6825","nonce":3814215935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6827","nonce":3814215935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6831","nonce":3814215935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6829","nonce":3814215935}]},"public_addr":"172.21.15.146:6825/3814215935","cluster_addr":"172.21.15.146:6827/3814215935","heartbeat_back_addr":"172.21.15.146:6831/3814215935","heartbeat_front_addr":"172.21.15.146:6829/3814215935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.820200+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.850189+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.764560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.866248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.807897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.869881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.782310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.827131+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:59:28.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.080+0000 7fbe24abf640 1 -- 172.21.15.69:0/2907102875 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fbe000744b0 msgr2=0x7fbe000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:28.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.080+0000 7fbe24abf640 1 --2- 172.21.15.69:0/2907102875 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fbe000744b0 0x7fbe000769c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbe14034000 tx=0x7fbe1402e000 comp rx=0 tx=0).stop 2024-04-25T10:59:28.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.080+0000 7fbe24abf640 1 -- 172.21.15.69:0/2907102875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe200d0e00 msgr2=0x7fbe200ca020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:28.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.080+0000 7fbe24abf640 1 --2- 172.21.15.69:0/2907102875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe200d0e00 0x7fbe200ca020 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbe1000c430 tx=0x7fbe1000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:59:28.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.080+0000 7fbe24abf640 1 -- 172.21.15.69:0/2907102875 shutdown_connections 2024-04-25T10:59:28.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.080+0000 7fbe24abf640 1 --2- 172.21.15.69:0/2907102875 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fbe000744b0 0x7fbe000769c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:28.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.080+0000 7fbe24abf640 1 --2- 172.21.15.69:0/2907102875 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbe200c97f0 0x7fbe200cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:28.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.080+0000 7fbe24abf640 1 --2- 172.21.15.69:0/2907102875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe200d4a50 0x7fbe200cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:28.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.080+0000 7fbe24abf640 1 --2- 172.21.15.69:0/2907102875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe200d0e00 0x7fbe200ca020 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:28.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.080+0000 7fbe24abf640 1 -- 172.21.15.69:0/2907102875 >> 172.21.15.69:0/2907102875 conn(0x7fbe200d5d90 msgr2=0x7fbe2006c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:28.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.080+0000 7fbe24abf640 1 -- 172.21.15.69:0/2907102875 shutdown_connections 2024-04-25T10:59:28.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.080+0000 7fbe24abf640 1 -- 172.21.15.69:0/2907102875 wait complete. 2024-04-25T10:59:28.092 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:59:06.422674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:59:16.727980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:59:17.938486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:59:28.093 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:59:28.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.202+0000 7f8068005640 1 Processor -- start 2024-04-25T10:59:28.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.202+0000 7f8068005640 1 -- start start 2024-04-25T10:59:28.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.204+0000 7f8068005640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f806006f3c0 0x7f806006f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:28.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.204+0000 7f8068005640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80600b1630 con 0x7f806015e4f0 2024-04-25T10:59:28.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.204+0000 7f8068005640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f806006fd10 con 0x7f806006cdc0 2024-04-25T10:59:28.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.204+0000 7f8068005640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f806006c930 con 0x7f806006f3c0 2024-04-25T10:59:28.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.204+0000 7f8065da1640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f806015e4f0 0x7f8060070070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:57210/0 (socket says 172.21.15.69:57210) 2024-04-25T10:59:28.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.204+0000 7f8065da1640 1 -- 172.21.15.69:0/3246179561 learned_addr learned my addr 172.21.15.69:0/3246179561 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:28.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.204+0000 7f80655a0640 1 --2- 172.21.15.69:0/3246179561 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f806006f3c0 0x7f806006f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:28.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.204+0000 7f8064d9f640 1 -- 172.21.15.69:0/3246179561 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2333799850 0 0) 0x7f805c001110 con 0x7f806015e4f0 2024-04-25T10:59:28.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.204+0000 7f8064d9f640 1 -- 172.21.15.69:0/3246179561 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f804c003520 con 0x7f806015e4f0 2024-04-25T10:59:28.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.204+0000 7f8064d9f640 1 -- 172.21.15.69:0/3246179561 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3920415400 0 0) 0x7f8058001490 con 0x7f806006cdc0 2024-04-25T10:59:28.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.204+0000 7f8064d9f640 1 -- 172.21.15.69:0/3246179561 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f805c001110 con 0x7f806006cdc0 2024-04-25T10:59:28.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.204+0000 7f80655a0640 1 -- 172.21.15.69:0/3246179561 >> v1:172.21.15.146:6789/0 conn(0x7f806006cdc0 legacy=0x7f806006d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:28.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.204+0000 7f80655a0640 1 -- 172.21.15.69:0/3246179561 >> v1:172.21.15.69:6789/0 conn(0x7f806015e4f0 legacy=0x7f8060070070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:28.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.204+0000 7f8064d9f640 1 -- 172.21.15.69:0/3246179561 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4123389212 0 0) 0x7f804c003520 con 0x7f806015e4f0 2024-04-25T10:59:28.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.204+0000 7f80655a0640 1 -- 172.21.15.69:0/3246179561 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80600d1440 con 0x7f806006f3c0 2024-04-25T10:59:28.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.205+0000 7f80655a0640 1 --2- 172.21.15.69:0/3246179561 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f806006f3c0 0x7f806006f7d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f80500097d0 tx=0x7f805002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc18410c884fa55d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:28.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.205+0000 7f8064d9f640 1 -- 172.21.15.69:0/3246179561 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8050037070 con 0x7f806006f3c0 2024-04-25T10:59:28.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.205+0000 7f8064d9f640 1 -- 172.21.15.69:0/3246179561 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f805002ab90 con 0x7f806006f3c0 2024-04-25T10:59:28.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.205+0000 7f8064d9f640 1 -- 172.21.15.69:0/3246179561 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c001840 con 0x7f806006f3c0 2024-04-25T10:59:28.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.206+0000 7f8068005640 1 -- 172.21.15.69:0/3246179561 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f806006f3c0 msgr2=0x7f806006f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:28.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.206+0000 7f8068005640 1 --2- 172.21.15.69:0/3246179561 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f806006f3c0 0x7f806006f7d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f80500097d0 tx=0x7f805002a100 comp rx=0 tx=0).stop 2024-04-25T10:59:28.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.206+0000 7f8068005640 1 -- 172.21.15.69:0/3246179561 shutdown_connections 2024-04-25T10:59:28.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.206+0000 7f8068005640 1 --2- 172.21.15.69:0/3246179561 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f806006f3c0 0x7f806006f7d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:28.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.206+0000 7f8068005640 1 -- 172.21.15.69:0/3246179561 >> 172.21.15.69:0/3246179561 conn(0x7f80600ce790 msgr2=0x7f80600ceba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:28.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.206+0000 7f8068005640 1 -- 172.21.15.69:0/3246179561 shutdown_connections 2024-04-25T10:59:28.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.206+0000 7f8068005640 1 -- 172.21.15.69:0/3246179561 wait complete. 2024-04-25T10:59:28.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.208+0000 7f8068005640 1 Processor -- start 2024-04-25T10:59:28.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.208+0000 7f8068005640 1 -- start start 2024-04-25T10:59:28.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.209+0000 7f8068005640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f806006cdc0 0x7f80601f9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:28.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.209+0000 7f8068005640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f806006f3c0 0x7f80600b6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:28.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.209+0000 7f8068005640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f806015e4f0 0x7f8060073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:28.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.209+0000 7f8068005640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f806006c5c0 con 0x7f806006f3c0 2024-04-25T10:59:28.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.209+0000 7f8068005640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f80600d1a00 con 0x7f806015e4f0 2024-04-25T10:59:28.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.209+0000 7f8068005640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f8060070780 con 0x7f806006cdc0 2024-04-25T10:59:28.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.209+0000 7f80655a0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f806006f3c0 0x7f80600b6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:28.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.209+0000 7f80655a0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f806006f3c0 0x7f80600b6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56380/0 (socket says 172.21.15.69:56380) 2024-04-25T10:59:28.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.209+0000 7f80655a0640 1 -- 172.21.15.69:0/1772791898 learned_addr learned my addr 172.21.15.69:0/1772791898 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:28.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.209+0000 7f8065da1640 1 --2- 172.21.15.69:0/1772791898 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f806006cdc0 0x7f80601f9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:28.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.209+0000 7f80655a0640 1 -- 172.21.15.69:0/1772791898 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f806006cdc0 msgr2=0x7f80601f9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:28.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.209+0000 7f80665a2640 1 --2- 172.21.15.69:0/1772791898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f806015e4f0 0x7f8060073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:28.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.209+0000 7f80655a0640 1 --2- 172.21.15.69:0/1772791898 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f806006cdc0 0x7f80601f9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:28.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.209+0000 7f80655a0640 1 -- 172.21.15.69:0/1772791898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f806015e4f0 msgr2=0x7f8060073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:28.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.209+0000 7f80655a0640 1 --2- 172.21.15.69:0/1772791898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f806015e4f0 0x7f8060073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:28.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.209+0000 7f80655a0640 1 -- 172.21.15.69:0/1772791898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80600cc400 con 0x7f806006f3c0 2024-04-25T10:59:28.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.210+0000 7f80655a0640 1 --2- 172.21.15.69:0/1772791898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f806006f3c0 0x7f80600b6e70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f80500097d0 tx=0x7f80500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:28.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.210+0000 7f80567fc640 1 -- 172.21.15.69:0/1772791898 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8050037040 con 0x7f806006f3c0 2024-04-25T10:59:28.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.210+0000 7f80567fc640 1 -- 172.21.15.69:0/1772791898 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8050004230 con 0x7f806006f3c0 2024-04-25T10:59:28.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.210+0000 7f80567fc640 1 -- 172.21.15.69:0/1772791898 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8050006e10 con 0x7f806006f3c0 2024-04-25T10:59:28.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.210+0000 7f8068005640 1 -- 172.21.15.69:0/1772791898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80600cc6e0 con 0x7f806006f3c0 2024-04-25T10:59:28.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.210+0000 7f8068005640 1 -- 172.21.15.69:0/1772791898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80600ccca0 con 0x7f806006f3c0 2024-04-25T10:59:28.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.211+0000 7f8068005640 1 -- 172.21.15.69:0/1772791898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8060050740 con 0x7f806006f3c0 2024-04-25T10:59:28.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.211+0000 7f80567fc640 1 -- 172.21.15.69:0/1772791898 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8050003610 con 0x7f806006f3c0 2024-04-25T10:59:28.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.212+0000 7f80567fc640 1 --2- 172.21.15.69:0/1772791898 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f80440744e0 0x7f80440769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:28.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.212+0000 7f80567fc640 1 -- 172.21.15.69:0/1772791898 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f80500b6030 con 0x7f806006f3c0 2024-04-25T10:59:28.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.212+0000 7f8065da1640 1 --2- 172.21.15.69:0/1772791898 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f80440744e0 0x7f80440769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:28.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.212+0000 7f8065da1640 1 --2- 172.21.15.69:0/1772791898 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f80440744e0 0x7f80440769f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f805c0013b0 tx=0x7f805c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:28.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.216+0000 7f80567fc640 1 -- 172.21.15.69:0/1772791898 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8050080850 con 0x7f806006f3c0 2024-04-25T10:59:28.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.394+0000 7f8068005640 1 -- 172.21.15.69:0/1772791898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f80600a27f0 con 0x7f806006f3c0 2024-04-25T10:59:28.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.395+0000 7f80567fc640 1 -- 172.21.15.69:0/1772791898 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f805003be10 con 0x7f806006f3c0 2024-04-25T10:59:28.395 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-25T10:59:28.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.396+0000 7f8068005640 1 -- 172.21.15.69:0/1772791898 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f80440744e0 msgr2=0x7f80440769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:28.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.396+0000 7f8068005640 1 --2- 172.21.15.69:0/1772791898 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f80440744e0 0x7f80440769f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f805c0013b0 tx=0x7f805c009040 comp rx=0 tx=0).stop 2024-04-25T10:59:28.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.396+0000 7f8068005640 1 -- 172.21.15.69:0/1772791898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f806006f3c0 msgr2=0x7f80600b6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:28.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.396+0000 7f8068005640 1 --2- 172.21.15.69:0/1772791898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f806006f3c0 0x7f80600b6e70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f80500097d0 tx=0x7f80500046c0 comp rx=0 tx=0).stop 2024-04-25T10:59:28.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.396+0000 7f8068005640 1 -- 172.21.15.69:0/1772791898 shutdown_connections 2024-04-25T10:59:28.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.396+0000 7f8068005640 1 --2- 172.21.15.69:0/1772791898 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f80440744e0 0x7f80440769f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:28.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.396+0000 7f8068005640 1 --2- 172.21.15.69:0/1772791898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f806015e4f0 0x7f8060073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:28.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.396+0000 7f8068005640 1 --2- 172.21.15.69:0/1772791898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f806006f3c0 0x7f80600b6e70 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:28.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.396+0000 7f8068005640 1 --2- 172.21.15.69:0/1772791898 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f806006cdc0 0x7f80601f9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:28.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.396+0000 7f8068005640 1 -- 172.21.15.69:0/1772791898 >> 172.21.15.69:0/1772791898 conn(0x7f80600ce790 msgr2=0x7f80600a0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:28.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.397+0000 7f8068005640 1 -- 172.21.15.69:0/1772791898 shutdown_connections 2024-04-25T10:59:28.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.397+0000 7f8068005640 1 -- 172.21.15.69:0/1772791898 wait complete. 2024-04-25T10:59:28.408 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:59:28.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.519+0000 7fa1f57ff640 1 Processor -- start 2024-04-25T10:59:28.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.519+0000 7fa1f57ff640 1 -- start start 2024-04-25T10:59:28.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.520+0000 7fa1f57ff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa1f00b4dc0 0x7fa1f00d52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:28.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.520+0000 7fa1f57ff640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa1f00b5280 0x7fa1f00d57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:28.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.520+0000 7fa1f57ff640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1f00d44e0 con 0x7fa1f00b9150 2024-04-25T10:59:28.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.520+0000 7fa1f57ff640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fa1f00d47c0 con 0x7fa1f00b5280 2024-04-25T10:59:28.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.520+0000 7fa1f57ff640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa1f00d4aa0 con 0x7fa1f00b4dc0 2024-04-25T10:59:28.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.520+0000 7fa1ef7fe640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fa1f00b9150 0x7fa1f00b9560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:57218/0 (socket says 172.21.15.69:57218) 2024-04-25T10:59:28.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.520+0000 7fa1ef7fe640 1 -- 172.21.15.69:0/4115813476 learned_addr learned my addr 172.21.15.69:0/4115813476 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:28.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.521+0000 7fa1eeffd640 1 --2- 172.21.15.69:0/4115813476 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa1f00b4dc0 0x7fa1f00d52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:28.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.521+0000 7fa1e7fff640 1 --2- 172.21.15.69:0/4115813476 >> v2:172.21.15.146:3300/0 conn(0x7fa1f00b5280 0x7fa1f00d57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:28.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.521+0000 7fa1ee7fc640 1 -- 172.21.15.69:0/4115813476 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3287421242 0 0) 0x7fa1f00d44e0 con 0x7fa1f00b9150 2024-04-25T10:59:28.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.521+0000 7fa1ee7fc640 1 -- 172.21.15.69:0/4115813476 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1dc003520 con 0x7fa1f00b9150 2024-04-25T10:59:28.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.521+0000 7fa1ee7fc640 1 -- 172.21.15.69:0/4115813476 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 305842821 0 0) 0x7fa1dc003520 con 0x7fa1f00b9150 2024-04-25T10:59:28.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.521+0000 7fa1ee7fc640 1 -- 172.21.15.69:0/4115813476 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa1f00b4dc0 msgr2=0x7fa1f00d52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:28.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.521+0000 7fa1ee7fc640 1 --2- 172.21.15.69:0/4115813476 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa1f00b4dc0 0x7fa1f00d52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:28.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.521+0000 7fa1ee7fc640 1 -- 172.21.15.69:0/4115813476 >> v2:172.21.15.146:3300/0 conn(0x7fa1f00b5280 msgr2=0x7fa1f00d57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:28.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.521+0000 7fa1ee7fc640 1 --2- 172.21.15.69:0/4115813476 >> v2:172.21.15.146:3300/0 conn(0x7fa1f00b5280 0x7fa1f00d57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:28.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.521+0000 7fa1ee7fc640 1 -- 172.21.15.69:0/4115813476 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f00d4d80 con 0x7fa1f00b9150 2024-04-25T10:59:28.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.521+0000 7fa1ee7fc640 1 -- 172.21.15.69:0/4115813476 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7fa1e80019f0 con 0x7fa1f00b9150 2024-04-25T10:59:28.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.521+0000 7fa1ee7fc640 1 -- 172.21.15.69:0/4115813476 >> v1:172.21.15.69:6789/0 conn(0x7fa1f00b9150 legacy=0x7fa1f00b9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:28.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.521+0000 7fa1ee7fc640 1 --2- 172.21.15.69:0/4115813476 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1dc0039b0 0x7fa1dc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:28.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.521+0000 7fa1ee7fc640 1 --2- 172.21.15.69:0/4115813476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1dc004800 0x7fa1dc004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:28.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.521+0000 7fa1ee7fc640 1 --2- 172.21.15.69:0/4115813476 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa1f00b4dc0 0x7fa1dc005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:28.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.521+0000 7fa1ee7fc640 1 -- 172.21.15.69:0/4115813476 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f00d47c0 con 0x7fa1dc004800 2024-04-25T10:59:28.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.521+0000 7fa1ee7fc640 1 -- 172.21.15.69:0/4115813476 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f00d4aa0 con 0x7fa1dc0039b0 2024-04-25T10:59:28.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.521+0000 7fa1ee7fc640 1 -- 172.21.15.69:0/4115813476 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa1dc00ba70 con 0x7fa1f00b4dc0 2024-04-25T10:59:28.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.521+0000 7fa1e7fff640 1 --2- 172.21.15.69:0/4115813476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1dc004800 0x7fa1dc004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:28.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.521+0000 7fa1ef7fe640 1 --2- 172.21.15.69:0/4115813476 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa1f00b4dc0 0x7fa1dc005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:28.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.521+0000 7fa1eeffd640 1 --2- 172.21.15.69:0/4115813476 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1dc0039b0 0x7fa1dc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:28.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.521+0000 7fa1e7fff640 1 -- 172.21.15.69:0/4115813476 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa1f00b4dc0 msgr2=0x7fa1dc005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:28.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.521+0000 7fa1e7fff640 1 --2- 172.21.15.69:0/4115813476 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa1f00b4dc0 0x7fa1dc005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:28.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.521+0000 7fa1e7fff640 1 -- 172.21.15.69:0/4115813476 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1dc0039b0 msgr2=0x7fa1dc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:28.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.521+0000 7fa1e7fff640 1 --2- 172.21.15.69:0/4115813476 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1dc0039b0 0x7fa1dc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:28.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.521+0000 7fa1e7fff640 1 -- 172.21.15.69:0/4115813476 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1dc003520 con 0x7fa1dc004800 2024-04-25T10:59:28.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.522+0000 7fa1e7fff640 1 --2- 172.21.15.69:0/4115813476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1dc004800 0x7fa1dc004c10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00d0380 tx=0x7fa1dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f717db72a234d630 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:28.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.522+0000 7fa1ee7fc640 1 -- 172.21.15.69:0/4115813476 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8036070 con 0x7fa1dc004800 2024-04-25T10:59:28.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.522+0000 7fa1ee7fc640 1 -- 172.21.15.69:0/4115813476 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1d8031400 con 0x7fa1dc004800 2024-04-25T10:59:28.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.522+0000 7fa1ee7fc640 1 -- 172.21.15.69:0/4115813476 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8009690 con 0x7fa1dc004800 2024-04-25T10:59:28.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.523+0000 7fa1f57ff640 1 -- 172.21.15.69:0/4115813476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1dc004800 msgr2=0x7fa1dc004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:28.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.523+0000 7fa1f57ff640 1 --2- 172.21.15.69:0/4115813476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1dc004800 0x7fa1dc004c10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00d0380 tx=0x7fa1dc005830 comp rx=0 tx=0).stop 2024-04-25T10:59:28.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.523+0000 7fa1ef7fe640 1 -- 172.21.15.69:0/4115813476 reap_dead start 2024-04-25T10:59:28.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.523+0000 7fa1f57ff640 1 -- 172.21.15.69:0/4115813476 shutdown_connections 2024-04-25T10:59:28.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.523+0000 7fa1f57ff640 1 -- 172.21.15.69:0/4115813476 >> 172.21.15.69:0/4115813476 conn(0x7fa1f00d2bf0 msgr2=0x7fa1f00d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:28.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.523+0000 7fa1f57ff640 1 -- 172.21.15.69:0/4115813476 shutdown_connections 2024-04-25T10:59:28.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.523+0000 7fa1f57ff640 1 -- 172.21.15.69:0/4115813476 wait complete. 2024-04-25T10:59:28.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.525+0000 7fa1f57ff640 1 Processor -- start 2024-04-25T10:59:28.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.525+0000 7fa1f57ff640 1 -- start start 2024-04-25T10:59:28.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.526+0000 7fa1f57ff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1dc004800 0x7fa1f00cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:28.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.526+0000 7fa1f57ff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa1f0162b50 0x7fa1f01f65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:28.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.526+0000 7fa1f57ff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1f01f6b00 0x7fa1f0161810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:28.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.526+0000 7fa1f57ff640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f00d0910 con 0x7fa1dc004800 2024-04-25T10:59:28.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.526+0000 7fa1f57ff640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f00d0790 con 0x7fa1f01f6b00 2024-04-25T10:59:28.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.526+0000 7fa1f57ff640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa1f0160450 con 0x7fa1f0162b50 2024-04-25T10:59:28.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.526+0000 7fa1eeffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1dc004800 0x7fa1f00cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:28.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.526+0000 7fa1eeffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1dc004800 0x7fa1f00cf8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56388/0 (socket says 172.21.15.69:56388) 2024-04-25T10:59:28.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.526+0000 7fa1eeffd640 1 -- 172.21.15.69:0/2777898580 learned_addr learned my addr 172.21.15.69:0/2777898580 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:28.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.526+0000 7fa1eeffd640 1 -- 172.21.15.69:0/2777898580 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa1f0162b50 msgr2=0x7fa1f01f65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:28.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.526+0000 7fa1eeffd640 1 --2- 172.21.15.69:0/2777898580 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa1f0162b50 0x7fa1f01f65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:28.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.526+0000 7fa1eeffd640 1 -- 172.21.15.69:0/2777898580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1f01f6b00 msgr2=0x7fa1f0161810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:28.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.526+0000 7fa1eeffd640 1 --2- 172.21.15.69:0/2777898580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1f01f6b00 0x7fa1f0161810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:28.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.526+0000 7fa1eeffd640 1 -- 172.21.15.69:0/2777898580 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f01f6f10 con 0x7fa1dc004800 2024-04-25T10:59:28.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.527+0000 7fa1eeffd640 1 --2- 172.21.15.69:0/2777898580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1dc004800 0x7fa1f00cf8e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00118e0 tx=0x7fa1e0011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:28.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.527+0000 7fa1e77fe640 1 -- 172.21.15.69:0/2777898580 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e000fa70 con 0x7fa1dc004800 2024-04-25T10:59:28.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.527+0000 7fa1e77fe640 1 -- 172.21.15.69:0/2777898580 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1e000fc10 con 0x7fa1dc004800 2024-04-25T10:59:28.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.527+0000 7fa1f57ff640 1 -- 172.21.15.69:0/2777898580 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f00ce840 con 0x7fa1dc004800 2024-04-25T10:59:28.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.527+0000 7fa1e77fe640 1 -- 172.21.15.69:0/2777898580 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0014440 con 0x7fa1dc004800 2024-04-25T10:59:28.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.527+0000 7fa1f57ff640 1 -- 172.21.15.69:0/2777898580 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f00cee00 con 0x7fa1dc004800 2024-04-25T10:59:28.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.528+0000 7fa1f57ff640 1 -- 172.21.15.69:0/2777898580 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1f0076c60 con 0x7fa1dc004800 2024-04-25T10:59:28.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.529+0000 7fa1e77fe640 1 -- 172.21.15.69:0/2777898580 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa1e0012040 con 0x7fa1dc004800 2024-04-25T10:59:28.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.529+0000 7fa1e77fe640 1 --2- 172.21.15.69:0/2777898580 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa1d00744b0 0x7fa1d00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:28.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.529+0000 7fa1e7fff640 1 --2- 172.21.15.69:0/2777898580 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa1d00744b0 0x7fa1d00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:28.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.529+0000 7fa1e77fe640 1 -- 172.21.15.69:0/2777898580 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa1e0095810 con 0x7fa1dc004800 2024-04-25T10:59:28.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.529+0000 7fa1e7fff640 1 --2- 172.21.15.69:0/2777898580 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa1d00744b0 0x7fa1d00769c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00d0380 tx=0x7fa1d802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:28.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.533+0000 7fa1e77fe640 1 -- 172.21.15.69:0/2777898580 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa1e0019030 con 0x7fa1dc004800 2024-04-25T10:59:28.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.710+0000 7fa1f57ff640 1 -- 172.21.15.69:0/2777898580 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa1f015c8f0 con 0x7fa1dc004800 2024-04-25T10:59:28.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.710+0000 7fa1e77fe640 1 -- 172.21.15.69:0/2777898580 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fa1e005ffa0 con 0x7fa1dc004800 2024-04-25T10:59:28.711 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-25T10:59:28.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.712+0000 7fa1f57ff640 1 -- 172.21.15.69:0/2777898580 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa1d00744b0 msgr2=0x7fa1d00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:28.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.712+0000 7fa1f57ff640 1 --2- 172.21.15.69:0/2777898580 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa1d00744b0 0x7fa1d00769c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00d0380 tx=0x7fa1d802e000 comp rx=0 tx=0).stop 2024-04-25T10:59:28.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.712+0000 7fa1f57ff640 1 -- 172.21.15.69:0/2777898580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1dc004800 msgr2=0x7fa1f00cf8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:28.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.712+0000 7fa1f57ff640 1 --2- 172.21.15.69:0/2777898580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1dc004800 0x7fa1f00cf8e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00118e0 tx=0x7fa1e0011da0 comp rx=0 tx=0).stop 2024-04-25T10:59:28.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.712+0000 7fa1f57ff640 1 -- 172.21.15.69:0/2777898580 shutdown_connections 2024-04-25T10:59:28.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.712+0000 7fa1f57ff640 1 --2- 172.21.15.69:0/2777898580 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa1d00744b0 0x7fa1d00769c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:28.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.712+0000 7fa1f57ff640 1 --2- 172.21.15.69:0/2777898580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1f01f6b00 0x7fa1f0161810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:28.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.712+0000 7fa1f57ff640 1 --2- 172.21.15.69:0/2777898580 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa1f0162b50 0x7fa1f01f65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:28.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.712+0000 7fa1f57ff640 1 --2- 172.21.15.69:0/2777898580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1dc004800 0x7fa1f00cf8e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:28.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.712+0000 7fa1f57ff640 1 -- 172.21.15.69:0/2777898580 >> 172.21.15.69:0/2777898580 conn(0x7fa1f00d2bf0 msgr2=0x7fa1f00b7a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:28.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.712+0000 7fa1f57ff640 1 -- 172.21.15.69:0/2777898580 shutdown_connections 2024-04-25T10:59:28.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.712+0000 7fa1f57ff640 1 -- 172.21.15.69:0/2777898580 wait complete. 2024-04-25T10:59:28.724 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:59:28.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.833+0000 7fb69c4ee640 1 Processor -- start 2024-04-25T10:59:28.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.833+0000 7fb69c4ee640 1 -- start start 2024-04-25T10:59:28.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.834+0000 7fb69c4ee640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb69408dec0 0x7fb6940b90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:28.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.834+0000 7fb69c4ee640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb69408cf20 0x7fb69408d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:28.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.834+0000 7fb69c4ee640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb69408d870 con 0x7fb6940b0c40 2024-04-25T10:59:28.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.834+0000 7fb69c4ee640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fb6940b95f0 con 0x7fb69408dec0 2024-04-25T10:59:28.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.834+0000 7fb69c4ee640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb6940d21e0 con 0x7fb69408cf20 2024-04-25T10:59:28.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.834+0000 7fb69aa8b640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fb6940b0c40 0x7fb6940b1050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:57228/0 (socket says 172.21.15.69:57228) 2024-04-25T10:59:28.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.834+0000 7fb69aa8b640 1 -- 172.21.15.69:0/3090200582 learned_addr learned my addr 172.21.15.69:0/3090200582 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:28.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.835+0000 7fb699a89640 1 --2- 172.21.15.69:0/3090200582 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb69408cf20 0x7fb69408d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:28.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.835+0000 7fb69a28a640 1 --2- 172.21.15.69:0/3090200582 >> v2:172.21.15.146:3300/0 conn(0x7fb69408dec0 0x7fb6940b90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:28.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.835+0000 7fb699288640 1 -- 172.21.15.69:0/3090200582 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1292538982 0 0) 0x7fb68c001490 con 0x7fb6940b0c40 2024-04-25T10:59:28.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.835+0000 7fb699288640 1 -- 172.21.15.69:0/3090200582 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb680003520 con 0x7fb6940b0c40 2024-04-25T10:59:28.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.835+0000 7fb699288640 1 -- 172.21.15.69:0/3090200582 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 324552266 0 0) 0x7fb680003520 con 0x7fb6940b0c40 2024-04-25T10:59:28.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.835+0000 7fb699288640 1 -- 172.21.15.69:0/3090200582 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb69408cf20 msgr2=0x7fb69408d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:28.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.835+0000 7fb699288640 1 --2- 172.21.15.69:0/3090200582 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb69408cf20 0x7fb69408d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:28.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.835+0000 7fb699288640 1 -- 172.21.15.69:0/3090200582 >> v2:172.21.15.146:3300/0 conn(0x7fb69408dec0 msgr2=0x7fb6940b90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:28.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.835+0000 7fb699288640 1 --2- 172.21.15.69:0/3090200582 >> v2:172.21.15.146:3300/0 conn(0x7fb69408dec0 0x7fb6940b90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:28.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.835+0000 7fb699288640 1 -- 172.21.15.69:0/3090200582 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6940d1940 con 0x7fb6940b0c40 2024-04-25T10:59:28.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.835+0000 7fb699288640 1 -- 172.21.15.69:0/3090200582 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7fb68c001bc0 con 0x7fb6940b0c40 2024-04-25T10:59:28.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.835+0000 7fb699288640 1 -- 172.21.15.69:0/3090200582 >> v1:172.21.15.69:6789/0 conn(0x7fb6940b0c40 legacy=0x7fb6940b1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:28.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.835+0000 7fb699288640 1 --2- 172.21.15.69:0/3090200582 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb69408cf20 0x7fb6800039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:28.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.835+0000 7fb699288640 1 --2- 172.21.15.69:0/3090200582 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb680003f30 0x7fb680004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:28.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.835+0000 7fb699288640 1 --2- 172.21.15.69:0/3090200582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb680004d40 0x7fb680005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:28.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.835+0000 7fb699288640 1 -- 172.21.15.69:0/3090200582 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb6940b95f0 con 0x7fb680004d40 2024-04-25T10:59:28.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.835+0000 7fb699288640 1 -- 172.21.15.69:0/3090200582 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb6940d21e0 con 0x7fb680003f30 2024-04-25T10:59:28.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.835+0000 7fb699288640 1 -- 172.21.15.69:0/3090200582 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb68000ba10 con 0x7fb69408cf20 2024-04-25T10:59:28.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.835+0000 7fb69aa8b640 1 --2- 172.21.15.69:0/3090200582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb680004d40 0x7fb680005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:28.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.835+0000 7fb69a28a640 1 --2- 172.21.15.69:0/3090200582 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb69408cf20 0x7fb6800039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:28.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.835+0000 7fb699a89640 1 --2- 172.21.15.69:0/3090200582 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb680003f30 0x7fb680004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:28.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.835+0000 7fb69aa8b640 1 -- 172.21.15.69:0/3090200582 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb69408cf20 msgr2=0x7fb6800039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:28.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.835+0000 7fb69aa8b640 1 --2- 172.21.15.69:0/3090200582 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb69408cf20 0x7fb6800039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:28.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.835+0000 7fb69aa8b640 1 -- 172.21.15.69:0/3090200582 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb680003f30 msgr2=0x7fb680004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:28.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.835+0000 7fb69aa8b640 1 --2- 172.21.15.69:0/3090200582 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb680003f30 0x7fb680004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:28.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.835+0000 7fb69aa8b640 1 -- 172.21.15.69:0/3090200582 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb680003520 con 0x7fb680004d40 2024-04-25T10:59:28.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.836+0000 7fb69aa8b640 1 --2- 172.21.15.69:0/3090200582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb680004d40 0x7fb680005150 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0013d0 tx=0x7fb680005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be856dcd59e8df0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:28.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.836+0000 7fb699288640 1 -- 172.21.15.69:0/3090200582 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c00a690 con 0x7fb680004d40 2024-04-25T10:59:28.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.836+0000 7fb699288640 1 -- 172.21.15.69:0/3090200582 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb68c02f070 con 0x7fb680004d40 2024-04-25T10:59:28.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.836+0000 7fb699288640 1 -- 172.21.15.69:0/3090200582 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c02be40 con 0x7fb680004d40 2024-04-25T10:59:28.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.837+0000 7fb69c4ee640 1 -- 172.21.15.69:0/3090200582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb680004d40 msgr2=0x7fb680005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:28.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.837+0000 7fb69c4ee640 1 --2- 172.21.15.69:0/3090200582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb680004d40 0x7fb680005150 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0013d0 tx=0x7fb680005830 comp rx=0 tx=0).stop 2024-04-25T10:59:28.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.837+0000 7fb69aa8b640 1 -- 172.21.15.69:0/3090200582 reap_dead start 2024-04-25T10:59:28.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.837+0000 7fb69c4ee640 1 -- 172.21.15.69:0/3090200582 shutdown_connections 2024-04-25T10:59:28.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.837+0000 7fb69c4ee640 1 -- 172.21.15.69:0/3090200582 >> 172.21.15.69:0/3090200582 conn(0x7fb6940d0510 msgr2=0x7fb6940d0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:28.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.837+0000 7fb69c4ee640 1 -- 172.21.15.69:0/3090200582 shutdown_connections 2024-04-25T10:59:28.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.837+0000 7fb69c4ee640 1 -- 172.21.15.69:0/3090200582 wait complete. 2024-04-25T10:59:28.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.839+0000 7fb69c4ee640 1 Processor -- start 2024-04-25T10:59:28.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.839+0000 7fb69c4ee640 1 -- start start 2024-04-25T10:59:28.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.840+0000 7fb69c4ee640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb680004d40 0x7fb69406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:28.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.840+0000 7fb69c4ee640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb69406dca0 0x7fb6940cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:28.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.840+0000 7fb69c4ee640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6940ccec0 0x7fb6940cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:28.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.840+0000 7fb69c4ee640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb6940b0720 con 0x7fb6940ccec0 2024-04-25T10:59:28.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.840+0000 7fb69c4ee640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb6940d1f00 con 0x7fb69406dca0 2024-04-25T10:59:28.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.840+0000 7fb69c4ee640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb6940d15d0 con 0x7fb680004d40 2024-04-25T10:59:28.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.840+0000 7fb69aa8b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6940ccec0 0x7fb6940cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:28.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.840+0000 7fb69aa8b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6940ccec0 0x7fb6940cd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56392/0 (socket says 172.21.15.69:56392) 2024-04-25T10:59:28.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.840+0000 7fb69aa8b640 1 -- 172.21.15.69:0/734708561 learned_addr learned my addr 172.21.15.69:0/734708561 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:28.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.840+0000 7fb69a28a640 1 --2- 172.21.15.69:0/734708561 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb680004d40 0x7fb69406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:28.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.840+0000 7fb699a89640 1 --2- 172.21.15.69:0/734708561 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb69406dca0 0x7fb6940cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:28.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.840+0000 7fb69aa8b640 1 -- 172.21.15.69:0/734708561 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb680004d40 msgr2=0x7fb69406d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:28.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.840+0000 7fb69aa8b640 1 --2- 172.21.15.69:0/734708561 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb680004d40 0x7fb69406d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:28.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.840+0000 7fb69aa8b640 1 -- 172.21.15.69:0/734708561 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb69406dca0 msgr2=0x7fb6940cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:28.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.840+0000 7fb69aa8b640 1 --2- 172.21.15.69:0/734708561 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb69406dca0 0x7fb6940cc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:28.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.840+0000 7fb69aa8b640 1 -- 172.21.15.69:0/734708561 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb69406e4e0 con 0x7fb6940ccec0 2024-04-25T10:59:28.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.840+0000 7fb699a89640 1 --2- 172.21.15.69:0/734708561 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb69406dca0 0x7fb6940cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:28.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.841+0000 7fb69aa8b640 1 --2- 172.21.15.69:0/734708561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6940ccec0 0x7fb6940cd2d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb68c000f80 tx=0x7fb6940d0330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:28.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.841+0000 7fb68affd640 1 -- 172.21.15.69:0/734708561 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c039070 con 0x7fb6940ccec0 2024-04-25T10:59:28.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.841+0000 7fb68affd640 1 -- 172.21.15.69:0/734708561 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb68c034670 con 0x7fb6940ccec0 2024-04-25T10:59:28.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.841+0000 7fb69c4ee640 1 -- 172.21.15.69:0/734708561 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb69406e7c0 con 0x7fb6940ccec0 2024-04-25T10:59:28.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.841+0000 7fb68affd640 1 -- 172.21.15.69:0/734708561 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c00a690 con 0x7fb6940ccec0 2024-04-25T10:59:28.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.841+0000 7fb69c4ee640 1 -- 172.21.15.69:0/734708561 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb69406ed80 con 0x7fb6940ccec0 2024-04-25T10:59:28.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.842+0000 7fb69c4ee640 1 -- 172.21.15.69:0/734708561 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb69406e250 con 0x7fb6940ccec0 2024-04-25T10:59:28.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.843+0000 7fb68affd640 1 -- 172.21.15.69:0/734708561 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb68c03d030 con 0x7fb6940ccec0 2024-04-25T10:59:28.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.843+0000 7fb68affd640 1 --2- 172.21.15.69:0/734708561 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb6780744b0 0x7fb6780769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:28.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.843+0000 7fb69a28a640 1 --2- 172.21.15.69:0/734708561 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb6780744b0 0x7fb6780769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:28.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.843+0000 7fb68affd640 1 -- 172.21.15.69:0/734708561 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb68c0b52f0 con 0x7fb6940ccec0 2024-04-25T10:59:28.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.843+0000 7fb69a28a640 1 --2- 172.21.15.69:0/734708561 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb6780744b0 0x7fb6780769c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb6940749e0 tx=0x7fb690009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:28.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:28.848+0000 7fb68affd640 1 -- 172.21.15.69:0/734708561 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb68c045060 con 0x7fb6940ccec0 2024-04-25T10:59:29.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.021+0000 7fb69c4ee640 1 -- 172.21.15.69:0/734708561 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb694050740 con 0x7fb6940ccec0 2024-04-25T10:59:29.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.022+0000 7fb68affd640 1 -- 172.21.15.69:0/734708561 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fb69408d870 con 0x7fb6940ccec0 2024-04-25T10:59:29.023 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-25T10:59:29.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.024+0000 7fb69c4ee640 1 -- 172.21.15.69:0/734708561 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb6780744b0 msgr2=0x7fb6780769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:29.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.024+0000 7fb69c4ee640 1 --2- 172.21.15.69:0/734708561 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb6780744b0 0x7fb6780769c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb6940749e0 tx=0x7fb690009190 comp rx=0 tx=0).stop 2024-04-25T10:59:29.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.024+0000 7fb69c4ee640 1 -- 172.21.15.69:0/734708561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6940ccec0 msgr2=0x7fb6940cd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:29.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.024+0000 7fb69c4ee640 1 --2- 172.21.15.69:0/734708561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6940ccec0 0x7fb6940cd2d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb68c000f80 tx=0x7fb6940d0330 comp rx=0 tx=0).stop 2024-04-25T10:59:29.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.024+0000 7fb69c4ee640 1 -- 172.21.15.69:0/734708561 shutdown_connections 2024-04-25T10:59:29.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.024+0000 7fb69c4ee640 1 --2- 172.21.15.69:0/734708561 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb6780744b0 0x7fb6780769c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:29.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.024+0000 7fb69c4ee640 1 --2- 172.21.15.69:0/734708561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6940ccec0 0x7fb6940cd2d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:29.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.024+0000 7fb69c4ee640 1 --2- 172.21.15.69:0/734708561 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb69406dca0 0x7fb6940cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:29.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.024+0000 7fb69c4ee640 1 --2- 172.21.15.69:0/734708561 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb680004d40 0x7fb69406d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:29.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.024+0000 7fb69c4ee640 1 -- 172.21.15.69:0/734708561 >> 172.21.15.69:0/734708561 conn(0x7fb6940d0510 msgr2=0x7fb694072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:29.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.024+0000 7fb69c4ee640 1 -- 172.21.15.69:0/734708561 shutdown_connections 2024-04-25T10:59:29.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.024+0000 7fb69c4ee640 1 -- 172.21.15.69:0/734708561 wait complete. 2024-04-25T10:59:29.035 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:59:29.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.142+0000 7fc1b0184640 1 Processor -- start 2024-04-25T10:59:29.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.142+0000 7fc1b0184640 1 -- start start 2024-04-25T10:59:29.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.144+0000 7fc1b0184640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fc1a80d1af0 0x7fc1a80d1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:29.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.144+0000 7fc1b0184640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1a80cf4b0 con 0x7fc1a80b23e0 2024-04-25T10:59:29.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.144+0000 7fc1b0184640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1a80cf790 con 0x7fc1a8161760 2024-04-25T10:59:29.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.144+0000 7fc1b0184640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fc1a80d00c0 con 0x7fc1a80d1af0 2024-04-25T10:59:29.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.144+0000 7fc1ad71f640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fc1a80d1af0 0x7fc1a80d1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:29.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.144+0000 7fc1ae721640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fc1a80b23e0 0x7fc1a80b27f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:57244/0 (socket says 172.21.15.69:57244) 2024-04-25T10:59:29.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.144+0000 7fc1ae721640 1 -- 172.21.15.69:0/1068972382 learned_addr learned my addr 172.21.15.69:0/1068972382 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:29.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.144+0000 7fc1ad71f640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fc1a80d1af0 0x7fc1a80d1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56408/0 (socket says 172.21.15.69:56408) 2024-04-25T10:59:29.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.144+0000 7fc1ad71f640 1 -- 172.21.15.69:0/1068972382 >> v1:172.21.15.146:6789/0 conn(0x7fc1a8161760 legacy=0x7fc1a80b7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:29.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.144+0000 7fc1ad71f640 1 -- 172.21.15.69:0/1068972382 >> v1:172.21.15.69:6789/0 conn(0x7fc1a80b23e0 legacy=0x7fc1a80b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:29.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.144+0000 7fc1acf1e640 1 -- 172.21.15.69:0/1068972382 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2450991020 0 0) 0x7fc1a80cf4b0 con 0x7fc1a80b23e0 2024-04-25T10:59:29.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.144+0000 7fc1ad71f640 1 -- 172.21.15.69:0/1068972382 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a80cfa70 con 0x7fc1a80d1af0 2024-04-25T10:59:29.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.145+0000 7fc1ad71f640 1 --2- 172.21.15.69:0/1068972382 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1a80d1af0 0x7fc1a80d1f00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc1980097a0 tx=0x7fc19802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e81023d35702d29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:29.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.145+0000 7fc1acf1e640 1 -- 172.21.15.69:0/1068972382 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc198037070 con 0x7fc1a80d1af0 2024-04-25T10:59:29.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.145+0000 7fc1acf1e640 1 -- 172.21.15.69:0/1068972382 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc198032440 con 0x7fc1a80d1af0 2024-04-25T10:59:29.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.145+0000 7fc1acf1e640 1 -- 172.21.15.69:0/1068972382 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc198032810 con 0x7fc1a80d1af0 2024-04-25T10:59:29.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.145+0000 7fc1b0184640 1 -- 172.21.15.69:0/1068972382 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1a80d1af0 msgr2=0x7fc1a80d1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:29.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.145+0000 7fc1b0184640 1 --2- 172.21.15.69:0/1068972382 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1a80d1af0 0x7fc1a80d1f00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc1980097a0 tx=0x7fc19802a230 comp rx=0 tx=0).stop 2024-04-25T10:59:29.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.146+0000 7fc1b0184640 1 -- 172.21.15.69:0/1068972382 shutdown_connections 2024-04-25T10:59:29.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.146+0000 7fc1b0184640 1 --2- 172.21.15.69:0/1068972382 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1a80d1af0 0x7fc1a80d1f00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:29.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.146+0000 7fc1b0184640 1 -- 172.21.15.69:0/1068972382 >> 172.21.15.69:0/1068972382 conn(0x7fc1a809b7d0 msgr2=0x7fc1a809bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:29.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.146+0000 7fc1b0184640 1 -- 172.21.15.69:0/1068972382 shutdown_connections 2024-04-25T10:59:29.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.146+0000 7fc1b0184640 1 -- 172.21.15.69:0/1068972382 wait complete. 2024-04-25T10:59:29.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.147+0000 7fc1b0184640 1 Processor -- start 2024-04-25T10:59:29.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.147+0000 7fc1b0184640 1 -- start start 2024-04-25T10:59:29.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.149+0000 7fc1b0184640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1a80b23e0 0x7fc1a8156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:29.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.149+0000 7fc1b0184640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1a80d1af0 0x7fc1a8159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:29.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.149+0000 7fc1b0184640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1a8161760 0x7fc1a815ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:29.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.149+0000 7fc1b0184640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a80b84a0 con 0x7fc1a80d1af0 2024-04-25T10:59:29.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.149+0000 7fc1b0184640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a8161b70 con 0x7fc1a80b23e0 2024-04-25T10:59:29.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.149+0000 7fc1b0184640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc1a80b7940 con 0x7fc1a8161760 2024-04-25T10:59:29.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.149+0000 7fc1ad71f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1a80d1af0 0x7fc1a8159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:29.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.149+0000 7fc1ad71f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1a80d1af0 0x7fc1a8159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56420/0 (socket says 172.21.15.69:56420) 2024-04-25T10:59:29.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.149+0000 7fc1ad71f640 1 -- 172.21.15.69:0/1356012721 learned_addr learned my addr 172.21.15.69:0/1356012721 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:29.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.149+0000 7fc1ae721640 1 --2- 172.21.15.69:0/1356012721 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1a8161760 0x7fc1a815ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:29.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.149+0000 7fc1adf20640 1 --2- 172.21.15.69:0/1356012721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1a80b23e0 0x7fc1a8156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:29.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.149+0000 7fc1ad71f640 1 -- 172.21.15.69:0/1356012721 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1a8161760 msgr2=0x7fc1a815ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:29.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.149+0000 7fc1ad71f640 1 --2- 172.21.15.69:0/1356012721 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1a8161760 0x7fc1a815ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:29.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.149+0000 7fc1ad71f640 1 -- 172.21.15.69:0/1356012721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1a80b23e0 msgr2=0x7fc1a8156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:29.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.149+0000 7fc1ad71f640 1 --2- 172.21.15.69:0/1356012721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1a80b23e0 0x7fc1a8156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:29.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.149+0000 7fc1ad71f640 1 -- 172.21.15.69:0/1356012721 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a81f4da0 con 0x7fc1a80d1af0 2024-04-25T10:59:29.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.149+0000 7fc1adf20640 1 --2- 172.21.15.69:0/1356012721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1a80b23e0 0x7fc1a8156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:29.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.149+0000 7fc1ad71f640 1 --2- 172.21.15.69:0/1356012721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1a80d1af0 0x7fc1a8159fb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc1980098c0 tx=0x7fc1980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:29.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.149+0000 7fc19e7fc640 1 -- 172.21.15.69:0/1356012721 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc198037040 con 0x7fc1a80d1af0 2024-04-25T10:59:29.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.149+0000 7fc19e7fc640 1 -- 172.21.15.69:0/1356012721 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc198042070 con 0x7fc1a80d1af0 2024-04-25T10:59:29.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.149+0000 7fc1b0184640 1 -- 172.21.15.69:0/1356012721 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a81f5010 con 0x7fc1a80d1af0 2024-04-25T10:59:29.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.149+0000 7fc19e7fc640 1 -- 172.21.15.69:0/1356012721 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc198030a70 con 0x7fc1a80d1af0 2024-04-25T10:59:29.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.149+0000 7fc1b0184640 1 -- 172.21.15.69:0/1356012721 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a81f55d0 con 0x7fc1a80d1af0 2024-04-25T10:59:29.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.150+0000 7fc1b0184640 1 -- 172.21.15.69:0/1356012721 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1a815aa90 con 0x7fc1a80d1af0 2024-04-25T10:59:29.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.151+0000 7fc19e7fc640 1 -- 172.21.15.69:0/1356012721 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc198030c10 con 0x7fc1a80d1af0 2024-04-25T10:59:29.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.151+0000 7fc19e7fc640 1 --2- 172.21.15.69:0/1356012721 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc18c0744b0 0x7fc18c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:29.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.152+0000 7fc19e7fc640 1 -- 172.21.15.69:0/1356012721 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc1980b5d40 con 0x7fc1a80d1af0 2024-04-25T10:59:29.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.152+0000 7fc1adf20640 1 --2- 172.21.15.69:0/1356012721 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc18c0744b0 0x7fc18c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:29.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.152+0000 7fc1adf20640 1 --2- 172.21.15.69:0/1356012721 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc18c0744b0 0x7fc18c0769c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc1a40012e0 tx=0x7fc1a4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:29.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.156+0000 7fc19e7fc640 1 -- 172.21.15.69:0/1356012721 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc1980804d0 con 0x7fc1a80d1af0 2024-04-25T10:59:29.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.368+0000 7fc1b0184640 1 -- 172.21.15.69:0/1356012721 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1a815cd20 con 0x7fc1a80d1af0 2024-04-25T10:59:29.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.368+0000 7fc19e7fc640 1 -- 172.21.15.69:0/1356012721 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5178 (secure 0 0 0) 0x7fc19807fdd0 con 0x7fc1a80d1af0 2024-04-25T10:59:29.369 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:29.369 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/2361834023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2361834023},{"type":"v1","addr":"172.21.15.146:6833","nonce":2361834023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1848276963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1848276963},{"type":"v1","addr":"172.21.15.69:6837","nonce":1848276963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1826742846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1826742846},{"type":"v1","addr":"172.21.15.69:6836","nonce":1826742846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:59:18.976232+0000","modified":"2024-04-25T10:59:20.983284+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6835/908008797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":908008797},{"type":"v1","addr":"172.21.15.146:6835","nonce":908008797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-04-25T10:59:29.370 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 6 2024-04-25T10:59:29.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.370+0000 7fc1b0184640 1 -- 172.21.15.69:0/1356012721 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc18c0744b0 msgr2=0x7fc18c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:29.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.370+0000 7fc1b0184640 1 --2- 172.21.15.69:0/1356012721 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc18c0744b0 0x7fc18c0769c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc1a40012e0 tx=0x7fc1a4009040 comp rx=0 tx=0).stop 2024-04-25T10:59:29.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.370+0000 7fc1b0184640 1 -- 172.21.15.69:0/1356012721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1a80d1af0 msgr2=0x7fc1a8159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:29.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.370+0000 7fc1b0184640 1 --2- 172.21.15.69:0/1356012721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1a80d1af0 0x7fc1a8159fb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc1980098c0 tx=0x7fc1980046c0 comp rx=0 tx=0).stop 2024-04-25T10:59:29.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.370+0000 7fc1b0184640 1 -- 172.21.15.69:0/1356012721 shutdown_connections 2024-04-25T10:59:29.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.370+0000 7fc1b0184640 1 --2- 172.21.15.69:0/1356012721 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc18c0744b0 0x7fc18c0769c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:29.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.370+0000 7fc1b0184640 1 --2- 172.21.15.69:0/1356012721 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1a8161760 0x7fc1a815ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:29.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.370+0000 7fc1b0184640 1 --2- 172.21.15.69:0/1356012721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1a80d1af0 0x7fc1a8159fb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:29.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.370+0000 7fc1b0184640 1 --2- 172.21.15.69:0/1356012721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1a80b23e0 0x7fc1a8156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:29.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.370+0000 7fc1b0184640 1 -- 172.21.15.69:0/1356012721 >> 172.21.15.69:0/1356012721 conn(0x7fc1a809b7d0 msgr2=0x7fc1a80d1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:29.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.371+0000 7fc1b0184640 1 -- 172.21.15.69:0/1356012721 shutdown_connections 2024-04-25T10:59:29.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.371+0000 7fc1b0184640 1 -- 172.21.15.69:0/1356012721 wait complete. 2024-04-25T10:59:29.382 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:59:29.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.489+0000 7fe8d37c0640 1 Processor -- start 2024-04-25T10:59:29.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.489+0000 7fe8d37c0640 1 -- start start 2024-04-25T10:59:29.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.491+0000 7fe8d37c0640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe8cc0b4830 0x7fe8cc06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:29.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.491+0000 7fe8d37c0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8cc0b2dc0 0x7fe8cc0b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:29.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.491+0000 7fe8d37c0640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8cc06d870 con 0x7fe8cc0d4570 2024-04-25T10:59:29.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.491+0000 7fe8d37c0640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fe8cc0b4c40 con 0x7fe8cc0b4830 2024-04-25T10:59:29.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.491+0000 7fe8d37c0640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe8cc06e460 con 0x7fe8cc0b2dc0 2024-04-25T10:59:29.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.491+0000 7fe8d155c640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fe8cc0d4570 0x7fe8cc0d04b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:57256/0 (socket says 172.21.15.69:57256) 2024-04-25T10:59:29.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.491+0000 7fe8d155c640 1 -- 172.21.15.69:0/4044370694 learned_addr learned my addr 172.21.15.69:0/4044370694 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:29.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.491+0000 7fe8d1d5d640 1 --2- 172.21.15.69:0/4044370694 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8cc0b2dc0 0x7fe8cc0b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:29.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.491+0000 7fe8d0d5b640 1 --2- 172.21.15.69:0/4044370694 >> v2:172.21.15.146:3300/0 conn(0x7fe8cc0b4830 0x7fe8cc06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:29.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.491+0000 7fe8c3fff640 1 -- 172.21.15.69:0/4044370694 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3840586790 0 0) 0x7fe8cc06d870 con 0x7fe8cc0d4570 2024-04-25T10:59:29.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.491+0000 7fe8c3fff640 1 -- 172.21.15.69:0/4044370694 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe8b8003520 con 0x7fe8cc0d4570 2024-04-25T10:59:29.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.491+0000 7fe8c3fff640 1 -- 172.21.15.69:0/4044370694 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3984001392 0 0) 0x7fe8b8003520 con 0x7fe8cc0d4570 2024-04-25T10:59:29.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.491+0000 7fe8c3fff640 1 -- 172.21.15.69:0/4044370694 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8cc0b2dc0 msgr2=0x7fe8cc0b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:29.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.491+0000 7fe8c3fff640 1 --2- 172.21.15.69:0/4044370694 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8cc0b2dc0 0x7fe8cc0b31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:29.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.491+0000 7fe8c3fff640 1 -- 172.21.15.69:0/4044370694 >> v2:172.21.15.146:3300/0 conn(0x7fe8cc0b4830 msgr2=0x7fe8cc06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:29.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.491+0000 7fe8c3fff640 1 --2- 172.21.15.69:0/4044370694 >> v2:172.21.15.146:3300/0 conn(0x7fe8cc0b4830 0x7fe8cc06e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:29.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.491+0000 7fe8c3fff640 1 -- 172.21.15.69:0/4044370694 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8cc06db10 con 0x7fe8cc0d4570 2024-04-25T10:59:29.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.491+0000 7fe8c3fff640 1 -- 172.21.15.69:0/4044370694 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7fe8c8001b70 con 0x7fe8cc0d4570 2024-04-25T10:59:29.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.491+0000 7fe8c3fff640 1 -- 172.21.15.69:0/4044370694 >> v1:172.21.15.69:6789/0 conn(0x7fe8cc0d4570 legacy=0x7fe8cc0d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:29.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.491+0000 7fe8c3fff640 1 --2- 172.21.15.69:0/4044370694 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8cc0b2dc0 0x7fe8b80039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:29.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.491+0000 7fe8c3fff640 1 --2- 172.21.15.69:0/4044370694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8b8003f50 0x7fe8b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:29.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.491+0000 7fe8c3fff640 1 --2- 172.21.15.69:0/4044370694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8b8004d40 0x7fe8b8005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:29.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.491+0000 7fe8c3fff640 1 -- 172.21.15.69:0/4044370694 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe8cc06e460 con 0x7fe8b8004d40 2024-04-25T10:59:29.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.491+0000 7fe8c3fff640 1 -- 172.21.15.69:0/4044370694 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b800ba70 con 0x7fe8b8003f50 2024-04-25T10:59:29.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.491+0000 7fe8c3fff640 1 -- 172.21.15.69:0/4044370694 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe8b800bc70 con 0x7fe8cc0b2dc0 2024-04-25T10:59:29.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.492+0000 7fe8d1d5d640 1 --2- 172.21.15.69:0/4044370694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8b8004d40 0x7fe8b8005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:29.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.492+0000 7fe8d0d5b640 1 --2- 172.21.15.69:0/4044370694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8b8003f50 0x7fe8b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:29.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.492+0000 7fe8d155c640 1 --2- 172.21.15.69:0/4044370694 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8cc0b2dc0 0x7fe8b80039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:29.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.492+0000 7fe8d1d5d640 1 -- 172.21.15.69:0/4044370694 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8cc0b2dc0 msgr2=0x7fe8b80039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:29.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.492+0000 7fe8d1d5d640 1 --2- 172.21.15.69:0/4044370694 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8cc0b2dc0 0x7fe8b80039d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:29.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.492+0000 7fe8d1d5d640 1 -- 172.21.15.69:0/4044370694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8b8003f50 msgr2=0x7fe8b8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:29.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.492+0000 7fe8d1d5d640 1 --2- 172.21.15.69:0/4044370694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8b8003f50 0x7fe8b8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:29.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.492+0000 7fe8d1d5d640 1 -- 172.21.15.69:0/4044370694 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b8003520 con 0x7fe8b8004d40 2024-04-25T10:59:29.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.493+0000 7fe8d1d5d640 1 --2- 172.21.15.69:0/4044370694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8b8004d40 0x7fe8b8005150 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4009ac0 tx=0x7fe8b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cdd076c72b210b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:29.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.493+0000 7fe8c3fff640 1 -- 172.21.15.69:0/4044370694 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4036070 con 0x7fe8b8004d40 2024-04-25T10:59:29.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.493+0000 7fe8c3fff640 1 -- 172.21.15.69:0/4044370694 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8c4031400 con 0x7fe8b8004d40 2024-04-25T10:59:29.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.493+0000 7fe8c3fff640 1 -- 172.21.15.69:0/4044370694 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4009690 con 0x7fe8b8004d40 2024-04-25T10:59:29.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.493+0000 7fe8d37c0640 1 -- 172.21.15.69:0/4044370694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8b8004d40 msgr2=0x7fe8b8005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:29.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.493+0000 7fe8d37c0640 1 --2- 172.21.15.69:0/4044370694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8b8004d40 0x7fe8b8005150 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4009ac0 tx=0x7fe8b8005830 comp rx=0 tx=0).stop 2024-04-25T10:59:29.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.493+0000 7fe8d1d5d640 1 -- 172.21.15.69:0/4044370694 reap_dead start 2024-04-25T10:59:29.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.493+0000 7fe8d37c0640 1 -- 172.21.15.69:0/4044370694 shutdown_connections 2024-04-25T10:59:29.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.493+0000 7fe8d37c0640 1 -- 172.21.15.69:0/4044370694 >> 172.21.15.69:0/4044370694 conn(0x7fe8cc0d3a50 msgr2=0x7fe8cc0d3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:29.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.494+0000 7fe8d37c0640 1 -- 172.21.15.69:0/4044370694 shutdown_connections 2024-04-25T10:59:29.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.494+0000 7fe8d37c0640 1 -- 172.21.15.69:0/4044370694 wait complete. 2024-04-25T10:59:29.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.495+0000 7fe8d37c0640 1 Processor -- start 2024-04-25T10:59:29.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.495+0000 7fe8d37c0640 1 -- start start 2024-04-25T10:59:29.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.496+0000 7fe8d37c0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8cc0b6db0 0x7fe8cc0b71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:29.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.497+0000 7fe8d37c0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8cc14a720 0x7fe8cc14ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:29.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.497+0000 7fe8d37c0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8cc14b070 0x7fe8cc1f9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:29.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.497+0000 7fe8d37c0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe8cc0d0fb0 con 0x7fe8cc0b6db0 2024-04-25T10:59:29.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.497+0000 7fe8d37c0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe8cc0d0bc0 con 0x7fe8cc14a720 2024-04-25T10:59:29.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.497+0000 7fe8d37c0640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe8cc0b31d0 con 0x7fe8cc14b070 2024-04-25T10:59:29.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.497+0000 7fe8d155c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8cc0b6db0 0x7fe8cc0b71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:29.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.497+0000 7fe8d155c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8cc0b6db0 0x7fe8cc0b71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56438/0 (socket says 172.21.15.69:56438) 2024-04-25T10:59:29.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.497+0000 7fe8d155c640 1 -- 172.21.15.69:0/2729833028 learned_addr learned my addr 172.21.15.69:0/2729833028 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:29.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.497+0000 7fe8d0d5b640 1 --2- 172.21.15.69:0/2729833028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8cc14a720 0x7fe8cc14ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:29.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.497+0000 7fe8d1d5d640 1 --2- 172.21.15.69:0/2729833028 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8cc14b070 0x7fe8cc1f9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:29.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.497+0000 7fe8d155c640 1 -- 172.21.15.69:0/2729833028 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8cc14b070 msgr2=0x7fe8cc1f9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:29.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.497+0000 7fe8d155c640 1 --2- 172.21.15.69:0/2729833028 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8cc14b070 0x7fe8cc1f9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:29.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.497+0000 7fe8d155c640 1 -- 172.21.15.69:0/2729833028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8cc14a720 msgr2=0x7fe8cc14ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:29.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.497+0000 7fe8d155c640 1 --2- 172.21.15.69:0/2729833028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8cc14a720 0x7fe8cc14ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:29.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.497+0000 7fe8d155c640 1 -- 172.21.15.69:0/2729833028 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8cc0b7890 con 0x7fe8cc0b6db0 2024-04-25T10:59:29.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.497+0000 7fe8d155c640 1 --2- 172.21.15.69:0/2729833028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8cc0b6db0 0x7fe8cc0b71c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8001a70 tx=0x7fe8c800d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:29.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.497+0000 7fe8c1ffb640 1 -- 172.21.15.69:0/2729833028 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c800de40 con 0x7fe8cc0b6db0 2024-04-25T10:59:29.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.497+0000 7fe8c1ffb640 1 -- 172.21.15.69:0/2729833028 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8c8018070 con 0x7fe8cc0b6db0 2024-04-25T10:59:29.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.497+0000 7fe8d37c0640 1 -- 172.21.15.69:0/2729833028 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8cc0b7b70 con 0x7fe8cc0b6db0 2024-04-25T10:59:29.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.497+0000 7fe8c1ffb640 1 -- 172.21.15.69:0/2729833028 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8005d40 con 0x7fe8cc0b6db0 2024-04-25T10:59:29.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.498+0000 7fe8d37c0640 1 -- 172.21.15.69:0/2729833028 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8cc0b8130 con 0x7fe8cc0b6db0 2024-04-25T10:59:29.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.498+0000 7fe8d37c0640 1 -- 172.21.15.69:0/2729833028 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8cc050740 con 0x7fe8cc0b6db0 2024-04-25T10:59:29.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.499+0000 7fe8c1ffb640 1 -- 172.21.15.69:0/2729833028 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe8c80070d0 con 0x7fe8cc0b6db0 2024-04-25T10:59:29.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.499+0000 7fe8c1ffb640 1 --2- 172.21.15.69:0/2729833028 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe8b00744b0 0x7fe8b00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:29.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.499+0000 7fe8c1ffb640 1 -- 172.21.15.69:0/2729833028 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe8c80956a0 con 0x7fe8cc0b6db0 2024-04-25T10:59:29.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.499+0000 7fe8d0d5b640 1 --2- 172.21.15.69:0/2729833028 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe8b00744b0 0x7fe8b00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:29.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.500+0000 7fe8d0d5b640 1 --2- 172.21.15.69:0/2729833028 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe8b00744b0 0x7fe8b00769c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc15b7c0 tx=0x7fe8bc009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:29.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.504+0000 7fe8c1ffb640 1 -- 172.21.15.69:0/2729833028 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe8c805fec0 con 0x7fe8cc0b6db0 2024-04-25T10:59:29.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.679+0000 7fe8d37c0640 1 -- 172.21.15.69:0/2729833028 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe8cc0d1220 con 0x7fe8cc0b6db0 2024-04-25T10:59:29.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.680+0000 7fe8c1ffb640 1 -- 172.21.15.69:0/2729833028 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7fe8c805f7c0 con 0x7fe8cc0b6db0 2024-04-25T10:59:29.681 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:29.681 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":20,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T10:59:20.630151+0000","last_up_change":"2024-04-25T10:59:05.546839+0000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:59:06.422674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:59:16.727980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:59:17.938486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6825","nonce":1876101065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6827","nonce":1876101065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6831","nonce":1876101065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6829","nonce":1876101065}]},"public_addr":"172.21.15.69:6825/1876101065","cluster_addr":"172.21.15.69:6827/1876101065","heartbeat_back_addr":"172.21.15.69:6831/1876101065","heartbeat_front_addr":"172.21.15.69:6829/1876101065","state":["exists","up"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6801","nonce":1806438563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6803","nonce":1806438563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6807","nonce":1806438563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6805","nonce":1806438563}]},"public_addr":"172.21.15.69:6801/1806438563","cluster_addr":"172.21.15.69:6803/1806438563","heartbeat_back_addr":"172.21.15.69:6807/1806438563","heartbeat_front_addr":"172.21.15.69:6805/1806438563","state":["exists","up"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6817","nonce":1593617983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6819","nonce":1593617983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6823","nonce":1593617983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6821","nonce":1593617983}]},"public_addr":"172.21.15.69:6817/1593617983","cluster_addr":"172.21.15.69:6819/1593617983","heartbeat_back_addr":"172.21.15.69:6823/1593617983","heartbeat_front_addr":"172.21.15.69:6821/1593617983","state":["exists","up"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6809","nonce":3950661294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6811","nonce":3950661294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6815","nonce":3950661294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6813","nonce":3950661294}]},"public_addr":"172.21.15.69:6809/3950661294","cluster_addr":"172.21.15.69:6811/3950661294","heartbeat_back_addr":"172.21.15.69:6815/3950661294","heartbeat_front_addr":"172.21.15.69:6813/3950661294","state":["exists","up"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6801","nonce":2508929872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6803","nonce":2508929872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6807","nonce":2508929872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6805","nonce":2508929872}]},"public_addr":"172.21.15.146:6801/2508929872","cluster_addr":"172.21.15.146:6803/2508929872","heartbeat_back_addr":"172.21.15.146:6807/2508929872","heartbeat_front_addr":"172.21.15.146:6805/2508929872","state":["exists","up"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6811","nonce":1310911138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6816","nonce":1310911138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6823","nonce":1310911138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6821","nonce":1310911138}]},"public_addr":"172.21.15.146:6811/1310911138","cluster_addr":"172.21.15.146:6816/1310911138","heartbeat_back_addr":"172.21.15.146:6823/1310911138","heartbeat_front_addr":"172.21.15.146:6821/1310911138","state":["exists","up"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6810","nonce":634688021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6813","nonce":634688021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6820","nonce":634688021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6817","nonce":634688021}]},"public_addr":"172.21.15.146:6810/634688021","cluster_addr":"172.21.15.146:6813/634688021","heartbeat_back_addr":"172.21.15.146:6820/634688021","heartbeat_front_addr":"172.21.15.146:6817/634688021","state":["exists","up"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6825","nonce":3814215935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6827","nonce":3814215935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6831","nonce":3814215935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6829","nonce":3814215935}]},"public_addr":"172.21.15.146:6825/3814215935","cluster_addr":"172.21.15.146:6827/3814215935","heartbeat_back_addr":"172.21.15.146:6831/3814215935","heartbeat_front_addr":"172.21.15.146:6829/3814215935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.820200+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.850189+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.764560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.866248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.807897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.869881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.782310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.827131+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:59:29.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.682+0000 7fe8d37c0640 1 -- 172.21.15.69:0/2729833028 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe8b00744b0 msgr2=0x7fe8b00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:29.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.682+0000 7fe8d37c0640 1 --2- 172.21.15.69:0/2729833028 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe8b00744b0 0x7fe8b00769c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc15b7c0 tx=0x7fe8bc009190 comp rx=0 tx=0).stop 2024-04-25T10:59:29.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.682+0000 7fe8d37c0640 1 -- 172.21.15.69:0/2729833028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8cc0b6db0 msgr2=0x7fe8cc0b71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:29.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.682+0000 7fe8d37c0640 1 --2- 172.21.15.69:0/2729833028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8cc0b6db0 0x7fe8cc0b71c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8001a70 tx=0x7fe8c800d520 comp rx=0 tx=0).stop 2024-04-25T10:59:29.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.682+0000 7fe8d37c0640 1 -- 172.21.15.69:0/2729833028 shutdown_connections 2024-04-25T10:59:29.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.682+0000 7fe8d37c0640 1 --2- 172.21.15.69:0/2729833028 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe8b00744b0 0x7fe8b00769c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:29.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.682+0000 7fe8d37c0640 1 --2- 172.21.15.69:0/2729833028 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8cc14b070 0x7fe8cc1f9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:29.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.682+0000 7fe8d37c0640 1 --2- 172.21.15.69:0/2729833028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8cc14a720 0x7fe8cc14ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:29.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.682+0000 7fe8d37c0640 1 --2- 172.21.15.69:0/2729833028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8cc0b6db0 0x7fe8cc0b71c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:29.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.682+0000 7fe8d37c0640 1 -- 172.21.15.69:0/2729833028 >> 172.21.15.69:0/2729833028 conn(0x7fe8cc0d3a50 msgr2=0x7fe8cc071bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:29.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.683+0000 7fe8d37c0640 1 -- 172.21.15.69:0/2729833028 shutdown_connections 2024-04-25T10:59:29.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.683+0000 7fe8d37c0640 1 -- 172.21.15.69:0/2729833028 wait complete. 2024-04-25T10:59:29.695 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:59:29.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.801+0000 7f2f1c7ab640 1 Processor -- start 2024-04-25T10:59:29.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.801+0000 7f2f1c7ab640 1 -- start start 2024-04-25T10:59:29.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.803+0000 7f2f1c7ab640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f140d1000 0x7f2f140d2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:29.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.803+0000 7f2f1c7ab640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2f140d25b0 0x7f2f140d3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:29.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.803+0000 7f2f1c7ab640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f140b0740 con 0x7f2f140d3cd0 2024-04-25T10:59:29.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.803+0000 7f2f1c7ab640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f2f140d2b30 con 0x7f2f140d25b0 2024-04-25T10:59:29.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.803+0000 7f2f1c7ab640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2f140b0a20 con 0x7f2f140d1000 2024-04-25T10:59:29.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.803+0000 7f2f1ad48640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f2f140d3cd0 0x7f2f140b38e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:57258/0 (socket says 172.21.15.69:57258) 2024-04-25T10:59:29.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.803+0000 7f2f1ad48640 1 -- 172.21.15.69:0/2559865507 learned_addr learned my addr 172.21.15.69:0/2559865507 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:29.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.803+0000 7f2f19d46640 1 --2- 172.21.15.69:0/2559865507 >> v2:172.21.15.146:3300/0 conn(0x7f2f140d25b0 0x7f2f140d3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:29.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.803+0000 7f2f1a547640 1 --2- 172.21.15.69:0/2559865507 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f140d1000 0x7f2f140d2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:29.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.803+0000 7f2f19545640 1 -- 172.21.15.69:0/2559865507 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2842401230 0 0) 0x7f2f140b0740 con 0x7f2f140d3cd0 2024-04-25T10:59:29.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.803+0000 7f2f19545640 1 -- 172.21.15.69:0/2559865507 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f00003520 con 0x7f2f140d3cd0 2024-04-25T10:59:29.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.803+0000 7f2f1a547640 1 -- 172.21.15.69:0/2559865507 >> v2:172.21.15.146:3300/0 conn(0x7f2f140d25b0 msgr2=0x7f2f140d3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:29.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.803+0000 7f2f1a547640 1 --2- 172.21.15.69:0/2559865507 >> v2:172.21.15.146:3300/0 conn(0x7f2f140d25b0 0x7f2f140d3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:29.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.803+0000 7f2f19545640 1 -- 172.21.15.69:0/2559865507 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2337475138 0 0) 0x7f2f00003520 con 0x7f2f140d3cd0 2024-04-25T10:59:29.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.803+0000 7f2f1a547640 1 -- 172.21.15.69:0/2559865507 >> v1:172.21.15.69:6789/0 conn(0x7f2f140d3cd0 legacy=0x7f2f140b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:29.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.803+0000 7f2f1a547640 1 -- 172.21.15.69:0/2559865507 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f140b0d00 con 0x7f2f140d1000 2024-04-25T10:59:29.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.803+0000 7f2f19d46640 1 --2- 172.21.15.69:0/2559865507 >> v2:172.21.15.146:3300/0 conn(0x7f2f140d25b0 0x7f2f140d3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:59:29.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.804+0000 7f2f1a547640 1 --2- 172.21.15.69:0/2559865507 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f140d1000 0x7f2f140d2070 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009d90 tx=0x7f2f1002a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87d2f587b6a1a688 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:29.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.804+0000 7f2f19545640 1 -- 172.21.15.69:0/2559865507 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c001670 con 0x7f2f140d1000 2024-04-25T10:59:29.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.804+0000 7f2f19545640 1 -- 172.21.15.69:0/2559865507 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f10007540 con 0x7f2f140d1000 2024-04-25T10:59:29.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.804+0000 7f2f19545640 1 -- 172.21.15.69:0/2559865507 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f100078f0 con 0x7f2f140d1000 2024-04-25T10:59:29.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.805+0000 7f2f1c7ab640 1 -- 172.21.15.69:0/2559865507 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f140d1000 msgr2=0x7f2f140d2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:29.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.805+0000 7f2f1c7ab640 1 --2- 172.21.15.69:0/2559865507 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f140d1000 0x7f2f140d2070 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009d90 tx=0x7f2f1002a5f0 comp rx=0 tx=0).stop 2024-04-25T10:59:29.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.805+0000 7f2f1c7ab640 1 -- 172.21.15.69:0/2559865507 shutdown_connections 2024-04-25T10:59:29.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.805+0000 7f2f1c7ab640 1 --2- 172.21.15.69:0/2559865507 >> v2:172.21.15.146:3300/0 conn(0x7f2f140d25b0 0x7f2f140d3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:29.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.805+0000 7f2f1c7ab640 1 --2- 172.21.15.69:0/2559865507 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f140d1000 0x7f2f140d2070 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:29.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.805+0000 7f2f1c7ab640 1 -- 172.21.15.69:0/2559865507 >> 172.21.15.69:0/2559865507 conn(0x7f2f1408d110 msgr2=0x7f2f1408d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:29.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.805+0000 7f2f1c7ab640 1 -- 172.21.15.69:0/2559865507 shutdown_connections 2024-04-25T10:59:29.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.805+0000 7f2f1c7ab640 1 -- 172.21.15.69:0/2559865507 wait complete. 2024-04-25T10:59:29.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.807+0000 7f2f1c7ab640 1 Processor -- start 2024-04-25T10:59:29.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.807+0000 7f2f1c7ab640 1 -- start start 2024-04-25T10:59:29.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.808+0000 7f2f1c7ab640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f140d1000 0x7f2f1414c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:29.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.808+0000 7f2f1c7ab640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f140d25b0 0x7f2f1414cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:29.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.808+0000 7f2f1c7ab640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f140d3cd0 0x7f2f140c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:29.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.808+0000 7f2f1c7ab640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2f140b2ad0 con 0x7f2f140d3cd0 2024-04-25T10:59:29.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.808+0000 7f2f1c7ab640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2f140d40e0 con 0x7f2f140d1000 2024-04-25T10:59:29.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.808+0000 7f2f1c7ab640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2f140b2510 con 0x7f2f140d25b0 2024-04-25T10:59:29.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.808+0000 7f2f1ad48640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f140d3cd0 0x7f2f140c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:29.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.808+0000 7f2f1ad48640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f140d3cd0 0x7f2f140c95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56450/0 (socket says 172.21.15.69:56450) 2024-04-25T10:59:29.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.808+0000 7f2f1ad48640 1 -- 172.21.15.69:0/365643027 learned_addr learned my addr 172.21.15.69:0/365643027 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:29.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.808+0000 7f2f19d46640 1 --2- 172.21.15.69:0/365643027 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f140d25b0 0x7f2f1414cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:29.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.808+0000 7f2f1a547640 1 --2- 172.21.15.69:0/365643027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f140d1000 0x7f2f1414c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:29.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.809+0000 7f2f1ad48640 1 -- 172.21.15.69:0/365643027 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f140d25b0 msgr2=0x7f2f1414cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:29.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.809+0000 7f2f1ad48640 1 --2- 172.21.15.69:0/365643027 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f140d25b0 0x7f2f1414cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:29.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.809+0000 7f2f1ad48640 1 -- 172.21.15.69:0/365643027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f140d1000 msgr2=0x7f2f1414c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:29.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.809+0000 7f2f1ad48640 1 --2- 172.21.15.69:0/365643027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f140d1000 0x7f2f1414c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:29.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.809+0000 7f2f1ad48640 1 -- 172.21.15.69:0/365643027 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f14153370 con 0x7f2f140d3cd0 2024-04-25T10:59:29.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.809+0000 7f2f1a547640 1 --2- 172.21.15.69:0/365643027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f140d1000 0x7f2f1414c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:29.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.809+0000 7f2f1ad48640 1 --2- 172.21.15.69:0/365643027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f140d3cd0 0x7f2f140c95e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00c520 tx=0x7f2f0c00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:29.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.809+0000 7f2f0affd640 1 -- 172.21.15.69:0/365643027 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c017070 con 0x7f2f140d3cd0 2024-04-25T10:59:29.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.809+0000 7f2f0affd640 1 -- 172.21.15.69:0/365643027 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f0c005ce0 con 0x7f2f140d3cd0 2024-04-25T10:59:29.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.809+0000 7f2f1c7ab640 1 -- 172.21.15.69:0/365643027 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f1414d660 con 0x7f2f140d3cd0 2024-04-25T10:59:29.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.809+0000 7f2f0affd640 1 -- 172.21.15.69:0/365643027 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c0072b0 con 0x7f2f140d3cd0 2024-04-25T10:59:29.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.809+0000 7f2f1c7ab640 1 -- 172.21.15.69:0/365643027 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f1414dc20 con 0x7f2f140d3cd0 2024-04-25T10:59:29.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.810+0000 7f2f1c7ab640 1 -- 172.21.15.69:0/365643027 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f14050740 con 0x7f2f140d3cd0 2024-04-25T10:59:29.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.811+0000 7f2f0affd640 1 -- 172.21.15.69:0/365643027 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2f0c01a070 con 0x7f2f140d3cd0 2024-04-25T10:59:29.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.811+0000 7f2f0affd640 1 --2- 172.21.15.69:0/365643027 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f2ef80744b0 0x7f2ef80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:29.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.811+0000 7f2f1a547640 1 --2- 172.21.15.69:0/365643027 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f2ef80744b0 0x7f2ef80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:29.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.811+0000 7f2f0affd640 1 -- 172.21.15.69:0/365643027 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2f0c094530 con 0x7f2f140d3cd0 2024-04-25T10:59:29.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.812+0000 7f2f1a547640 1 --2- 172.21.15.69:0/365643027 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f2ef80744b0 0x7f2ef80769c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2f10034000 tx=0x7f2f10033040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:29.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:29.816+0000 7f2f0affd640 1 -- 172.21.15.69:0/365643027 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2f0c05ed50 con 0x7f2f140d3cd0 2024-04-25T10:59:30.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.027+0000 7f2f1c7ab640 1 -- 172.21.15.69:0/365643027 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f14068780 con 0x7f2f140d3cd0 2024-04-25T10:59:30.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.027+0000 7f2f0affd640 1 -- 172.21.15.69:0/365643027 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5178 (secure 0 0 0) 0x7f2f0c05e650 con 0x7f2f140d3cd0 2024-04-25T10:59:30.028 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:30.028 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/2361834023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2361834023},{"type":"v1","addr":"172.21.15.146:6833","nonce":2361834023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1848276963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1848276963},{"type":"v1","addr":"172.21.15.69:6837","nonce":1848276963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1826742846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1826742846},{"type":"v1","addr":"172.21.15.69:6836","nonce":1826742846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:59:18.976232+0000","modified":"2024-04-25T10:59:20.983284+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6835/908008797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":908008797},{"type":"v1","addr":"172.21.15.146:6835","nonce":908008797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-04-25T10:59:30.028 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 6 2024-04-25T10:59:30.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.029+0000 7f2f1c7ab640 1 -- 172.21.15.69:0/365643027 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f2ef80744b0 msgr2=0x7f2ef80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:30.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.029+0000 7f2f1c7ab640 1 --2- 172.21.15.69:0/365643027 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f2ef80744b0 0x7f2ef80769c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2f10034000 tx=0x7f2f10033040 comp rx=0 tx=0).stop 2024-04-25T10:59:30.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.029+0000 7f2f1c7ab640 1 -- 172.21.15.69:0/365643027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f140d3cd0 msgr2=0x7f2f140c95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:30.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.029+0000 7f2f1c7ab640 1 --2- 172.21.15.69:0/365643027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f140d3cd0 0x7f2f140c95e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00c520 tx=0x7f2f0c00c9e0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.029+0000 7f2f1c7ab640 1 -- 172.21.15.69:0/365643027 shutdown_connections 2024-04-25T10:59:30.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.029+0000 7f2f1c7ab640 1 --2- 172.21.15.69:0/365643027 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f2ef80744b0 0x7f2ef80769c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.029+0000 7f2f1c7ab640 1 --2- 172.21.15.69:0/365643027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f140d3cd0 0x7f2f140c95e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.029+0000 7f2f1c7ab640 1 --2- 172.21.15.69:0/365643027 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f140d25b0 0x7f2f1414cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.029+0000 7f2f1c7ab640 1 --2- 172.21.15.69:0/365643027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f140d1000 0x7f2f1414c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.029+0000 7f2f1c7ab640 1 -- 172.21.15.69:0/365643027 >> 172.21.15.69:0/365643027 conn(0x7f2f1408d110 msgr2=0x7f2f140b5580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:30.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.029+0000 7f2f1c7ab640 1 -- 172.21.15.69:0/365643027 shutdown_connections 2024-04-25T10:59:30.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.029+0000 7f2f1c7ab640 1 -- 172.21.15.69:0/365643027 wait complete. 2024-04-25T10:59:30.446 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:59:18.976232+0000', 'modified': '2024-04-25T10:59:20.983284+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4294}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4294': {'gid': 4294, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.146:6835/908008797', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6834', 'nonce': 908008797}, {'type': 'v1', 'addr': '172.21.15.146:6835', 'nonce': 908008797}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4294, 'qdb_cluster': [4294]} 2024-04-25T10:59:30.447 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:59:30.447 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T10:59:30.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.555+0000 7f7492d14640 1 Processor -- start 2024-04-25T10:59:30.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.555+0000 7f7492d14640 1 -- start start 2024-04-25T10:59:30.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.557+0000 7f7492d14640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f748c164b00 0x7f748c0b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:30.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.557+0000 7f7492d14640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f748c160fa0 0x7f748c0b80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:30.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.557+0000 7f7492d14640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f748c1615f0 con 0x7f748c15df80 2024-04-25T10:59:30.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.557+0000 7f7492d14640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f748c1613b0 con 0x7f748c160fa0 2024-04-25T10:59:30.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.557+0000 7f7492d14640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f748c164670 con 0x7f748c164b00 2024-04-25T10:59:30.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.557+0000 7f74912b1640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f748c160fa0 0x7f748c0b80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:30.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.557+0000 7f74912b1640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f748c160fa0 0x7f748c0b80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56458/0 (socket says 172.21.15.69:56458) 2024-04-25T10:59:30.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.557+0000 7f74912b1640 1 -- 172.21.15.69:0/1527193613 learned_addr learned my addr 172.21.15.69:0/1527193613 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:30.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.557+0000 7f7490ab0640 1 --2- 172.21.15.69:0/1527193613 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f748c164b00 0x7f748c0b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:30.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.557+0000 7f74912b1640 1 -- 172.21.15.69:0/1527193613 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f748c164b00 msgr2=0x7f748c0b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:30.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.557+0000 7f74912b1640 1 --2- 172.21.15.69:0/1527193613 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f748c164b00 0x7f748c0b7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.557+0000 7f74912b1640 1 -- 172.21.15.69:0/1527193613 >> v1:172.21.15.146:6789/0 conn(0x7f748c15df80 legacy=0x7f748c160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:30.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.557+0000 7f74912b1640 1 -- 172.21.15.69:0/1527193613 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f748c164020 con 0x7f748c160fa0 2024-04-25T10:59:30.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.558+0000 7f74912b1640 1 --2- 172.21.15.69:0/1527193613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f748c160fa0 0x7f748c0b80c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7484009e50 tx=0x7f748402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48dc7dc26418330f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:30.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.558+0000 7f748b7fe640 1 -- 172.21.15.69:0/1527193613 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7484036070 con 0x7f748c160fa0 2024-04-25T10:59:30.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.558+0000 7f748b7fe640 1 -- 172.21.15.69:0/1527193613 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7484031600 con 0x7f748c160fa0 2024-04-25T10:59:30.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.558+0000 7f748b7fe640 1 -- 172.21.15.69:0/1527193613 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74840319d0 con 0x7f748c160fa0 2024-04-25T10:59:30.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.559+0000 7f7492d14640 1 -- 172.21.15.69:0/1527193613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f748c160fa0 msgr2=0x7f748c0b80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:30.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.559+0000 7f7492d14640 1 --2- 172.21.15.69:0/1527193613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f748c160fa0 0x7f748c0b80c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7484009e50 tx=0x7f748402a710 comp rx=0 tx=0).stop 2024-04-25T10:59:30.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.559+0000 7f7492d14640 1 -- 172.21.15.69:0/1527193613 shutdown_connections 2024-04-25T10:59:30.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.559+0000 7f7492d14640 1 --2- 172.21.15.69:0/1527193613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f748c160fa0 0x7f748c0b80c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.559+0000 7f7492d14640 1 --2- 172.21.15.69:0/1527193613 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f748c164b00 0x7f748c0b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.559+0000 7f7492d14640 1 -- 172.21.15.69:0/1527193613 >> 172.21.15.69:0/1527193613 conn(0x7f748c0d2bc0 msgr2=0x7f748c0d2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:30.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.559+0000 7f7492d14640 1 -- 172.21.15.69:0/1527193613 shutdown_connections 2024-04-25T10:59:30.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.559+0000 7f7492d14640 1 -- 172.21.15.69:0/1527193613 wait complete. 2024-04-25T10:59:30.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.560+0000 7f7492d14640 1 Processor -- start 2024-04-25T10:59:30.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.561+0000 7f7492d14640 1 -- start start 2024-04-25T10:59:30.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.562+0000 7f7492d14640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f748c15df80 0x7f748c1521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:30.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.562+0000 7f7492d14640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f748c160fa0 0x7f748c1555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:30.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.562+0000 7f7492d14640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f748c164b00 0x7f748c155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:30.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.562+0000 7f7492d14640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f748c0b4a00 con 0x7f748c160fa0 2024-04-25T10:59:30.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.562+0000 7f7492d14640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f748c0b4880 con 0x7f748c164b00 2024-04-25T10:59:30.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.562+0000 7f7492d14640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f748c0b4b80 con 0x7f748c15df80 2024-04-25T10:59:30.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.562+0000 7f748bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f748c160fa0 0x7f748c1555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:30.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.562+0000 7f748bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f748c160fa0 0x7f748c1555d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56460/0 (socket says 172.21.15.69:56460) 2024-04-25T10:59:30.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.562+0000 7f748bfff640 1 -- 172.21.15.69:0/1440922993 learned_addr learned my addr 172.21.15.69:0/1440922993 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:30.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.562+0000 7f7490ab0640 1 --2- 172.21.15.69:0/1440922993 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f748c15df80 0x7f748c1521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:30.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.562+0000 7f74912b1640 1 --2- 172.21.15.69:0/1440922993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f748c164b00 0x7f748c155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:30.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.562+0000 7f748bfff640 1 -- 172.21.15.69:0/1440922993 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f748c15df80 msgr2=0x7f748c1521f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:30.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.562+0000 7f748bfff640 1 --2- 172.21.15.69:0/1440922993 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f748c15df80 0x7f748c1521f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.562+0000 7f748bfff640 1 -- 172.21.15.69:0/1440922993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f748c164b00 msgr2=0x7f748c155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:30.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.562+0000 7f748bfff640 1 --2- 172.21.15.69:0/1440922993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f748c164b00 0x7f748c155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.562+0000 7f748bfff640 1 -- 172.21.15.69:0/1440922993 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f748c159b90 con 0x7f748c160fa0 2024-04-25T10:59:30.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.562+0000 7f74912b1640 1 --2- 172.21.15.69:0/1440922993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f748c164b00 0x7f748c155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:30.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.563+0000 7f748bfff640 1 --2- 172.21.15.69:0/1440922993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f748c160fa0 0x7f748c1555d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f748000c3f0 tx=0x7f748000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:30.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.563+0000 7f74897fa640 1 -- 172.21.15.69:0/1440922993 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74800052d0 con 0x7f748c160fa0 2024-04-25T10:59:30.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.563+0000 7f74897fa640 1 -- 172.21.15.69:0/1440922993 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7480005490 con 0x7f748c160fa0 2024-04-25T10:59:30.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.563+0000 7f74897fa640 1 -- 172.21.15.69:0/1440922993 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7480007260 con 0x7f748c160fa0 2024-04-25T10:59:30.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.563+0000 7f7492d14640 1 -- 172.21.15.69:0/1440922993 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f748c159e70 con 0x7f748c160fa0 2024-04-25T10:59:30.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.563+0000 7f7492d14640 1 -- 172.21.15.69:0/1440922993 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f748c15a430 con 0x7f748c160fa0 2024-04-25T10:59:30.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.564+0000 7f74897fa640 1 -- 172.21.15.69:0/1440922993 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7480007400 con 0x7f748c160fa0 2024-04-25T10:59:30.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.565+0000 7f74897fa640 1 --2- 172.21.15.69:0/1440922993 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f74780744b0 0x7f74780769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:30.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.565+0000 7f7490ab0640 1 --2- 172.21.15.69:0/1440922993 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f74780744b0 0x7f74780769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:30.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.565+0000 7f74897fa640 1 -- 172.21.15.69:0/1440922993 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7480094600 con 0x7f748c160fa0 2024-04-25T10:59:30.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.565+0000 7f7490ab0640 1 --2- 172.21.15.69:0/1440922993 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f74780744b0 0x7f74780769c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f747c009a30 tx=0x7f747c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:30.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.566+0000 7f7499bc8740 1 Processor -- start 2024-04-25T10:59:30.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.566+0000 7f7499bc8740 1 -- start start 2024-04-25T10:59:30.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.567+0000 7f7499bc8740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfd50d6270 0x55bfd54a2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:30.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.567+0000 7f7499bc8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfd50ad3b0 0x55bfd549efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:30.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.567+0000 7f7499bc8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd50b5590 0x55bfd5478260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:30.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.567+0000 7f7499bc8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bfd53dc740 con 0x55bfd50b5590 2024-04-25T10:59:30.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.567+0000 7f7499bc8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55bfd53783a0 con 0x55bfd50ad3b0 2024-04-25T10:59:30.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.567+0000 7f7499bc8740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x55bfd53d8700 con 0x55bfd50d6270 2024-04-25T10:59:30.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.567+0000 7f74912b1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd50b5590 0x55bfd5478260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:30.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.567+0000 7f74912b1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd50b5590 0x55bfd5478260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56474/0 (socket says 172.21.15.69:56474) 2024-04-25T10:59:30.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.567+0000 7f74912b1640 1 -- 172.21.15.69:0/1735637074 learned_addr learned my addr 172.21.15.69:0/1735637074 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:30.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.567+0000 7f74912b1640 1 -- 172.21.15.69:0/1735637074 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfd50d6270 msgr2=0x55bfd54a2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:30.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.567+0000 7f7490ab0640 1 --2- 172.21.15.69:0/1735637074 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfd50d6270 0x55bfd54a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:30.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.567+0000 7f748bfff640 1 --2- 172.21.15.69:0/1735637074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfd50ad3b0 0x55bfd549efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:30.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.567+0000 7f74912b1640 1 --2- 172.21.15.69:0/1735637074 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfd50d6270 0x55bfd54a2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.567+0000 7f74912b1640 1 -- 172.21.15.69:0/1735637074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfd50ad3b0 msgr2=0x55bfd549efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:30.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.567+0000 7f74912b1640 1 --2- 172.21.15.69:0/1735637074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfd50ad3b0 0x55bfd549efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.567+0000 7f74912b1640 1 -- 172.21.15.69:0/1735637074 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfd5394750 con 0x55bfd50b5590 2024-04-25T10:59:30.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.568+0000 7f74912b1640 1 --2- 172.21.15.69:0/1735637074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd50b5590 0x55bfd5478260 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7484009f70 tx=0x7f748402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806db61c43e9475f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:30.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.568+0000 7f74727fc640 1 -- 172.21.15.69:0/1735637074 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7484036040 con 0x55bfd50b5590 2024-04-25T10:59:30.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.568+0000 7f74727fc640 1 -- 172.21.15.69:0/1735637074 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7484042070 con 0x55bfd50b5590 2024-04-25T10:59:30.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.568+0000 7f74727fc640 1 -- 172.21.15.69:0/1735637074 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748403c870 con 0x55bfd50b5590 2024-04-25T10:59:30.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.568+0000 7f7499bc8740 1 -- 172.21.15.69:0/1735637074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd50b5590 msgr2=0x55bfd5478260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:30.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.568+0000 7f7499bc8740 1 --2- 172.21.15.69:0/1735637074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd50b5590 0x55bfd5478260 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7484009f70 tx=0x7f748402e000 comp rx=0 tx=0).stop 2024-04-25T10:59:30.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.568+0000 7f7499bc8740 1 -- 172.21.15.69:0/1735637074 shutdown_connections 2024-04-25T10:59:30.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.568+0000 7f7499bc8740 1 --2- 172.21.15.69:0/1735637074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd50b5590 0x55bfd5478260 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.568+0000 7f7499bc8740 1 --2- 172.21.15.69:0/1735637074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfd50ad3b0 0x55bfd549efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.568+0000 7f7499bc8740 1 --2- 172.21.15.69:0/1735637074 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfd50d6270 0x55bfd54a2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.568+0000 7f7499bc8740 1 -- 172.21.15.69:0/1735637074 >> 172.21.15.69:0/1735637074 conn(0x55bfd50ca3c0 msgr2=0x55bfd54779f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:30.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.569+0000 7f7499bc8740 1 -- 172.21.15.69:0/1735637074 shutdown_connections 2024-04-25T10:59:30.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.569+0000 7f7499bc8740 1 -- 172.21.15.69:0/1735637074 wait complete. 2024-04-25T10:59:30.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.569+0000 7f7499bc8740 1 -- start start 2024-04-25T10:59:30.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.570+0000 7f7499bc8740 1 Processor -- start 2024-04-25T10:59:30.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.571+0000 7f7499bc8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd50ad3b0 0x55bfd549b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:30.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.571+0000 7f7499bc8740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfd50b5590 0x55bfd54b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:30.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.571+0000 7f7499bc8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfd50d6270 0x55bfd549e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:30.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.571+0000 7f7499bc8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bfd5363fe0 con 0x55bfd50ad3b0 2024-04-25T10:59:30.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.571+0000 7f7499bc8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55bfd54b46f0 con 0x55bfd50d6270 2024-04-25T10:59:30.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.571+0000 7f7499bc8740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x55bfd54b4890 con 0x55bfd50b5590 2024-04-25T10:59:30.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.571+0000 7f7499bc8740 10 client.0 _ll_register_callbacks cb 0x55bfd537a150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:59:30.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.571+0000 7f7490ab0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd50ad3b0 0x55bfd549b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:30.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.571+0000 7f7490ab0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd50ad3b0 0x55bfd549b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56478/0 (socket says 172.21.15.69:56478) 2024-04-25T10:59:30.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.571+0000 7f7490ab0640 1 -- 172.21.15.69:0/2677050469 learned_addr learned my addr 172.21.15.69:0/2677050469 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:30.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.571+0000 7f7490ab0640 1 -- 172.21.15.69:0/2677050469 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfd50b5590 msgr2=0x55bfd54b41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:30.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.571+0000 7f7490ab0640 1 --2- 172.21.15.69:0/2677050469 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfd50b5590 0x55bfd54b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.571+0000 7f7490ab0640 1 -- 172.21.15.69:0/2677050469 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfd50d6270 msgr2=0x55bfd549e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:30.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.571+0000 7f74912b1640 1 --2- 172.21.15.69:0/2677050469 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfd50d6270 0x55bfd549e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:30.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.571+0000 7f7490ab0640 1 --2- 172.21.15.69:0/2677050469 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfd50d6270 0x55bfd549e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.571+0000 7f7490ab0640 1 -- 172.21.15.69:0/2677050469 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfd50ee2d0 con 0x55bfd50ad3b0 2024-04-25T10:59:30.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.571+0000 7f7490ab0640 1 --2- 172.21.15.69:0/2677050469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd50ad3b0 0x55bfd549b1a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f747c0156c0 tx=0x7f747c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b617eaf6e27c8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:30.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.571+0000 7f746b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-25T10:59:30.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.572+0000 7f746b7fe640 1 -- 172.21.15.69:0/2677050469 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c025070 con 0x55bfd50ad3b0 2024-04-25T10:59:30.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.572+0000 7f746b7fe640 1 -- 172.21.15.69:0/2677050469 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f747c0146b0 con 0x55bfd50ad3b0 2024-04-25T10:59:30.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.572+0000 7f7499bc8740 1 -- 172.21.15.69:0/2677050469 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7484042070 con 0x55bfd50ad3b0 2024-04-25T10:59:30.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.572+0000 7f746b7fe640 1 -- 172.21.15.69:0/2677050469 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7f747c030050 con 0x55bfd50ad3b0 2024-04-25T10:59:30.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.572+0000 7f7499bc8740 10 client.4626 fetch_fsmap learned FSMap version 6 2024-04-25T10:59:30.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.572+0000 7f7499bc8740 1 -- 172.21.15.69:0/2677050469 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55bfd51b86b0 con 0x55bfd50ad3b0 2024-04-25T10:59:30.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.572+0000 7f746b7fe640 1 -- 172.21.15.69:0/2677050469 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7484042070 con 0x55bfd50ad3b0 2024-04-25T10:59:30.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.572+0000 7f746b7fe640 1 -- 172.21.15.69:0/2677050469 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 6) ==== 3494+0+0 (secure 0 0 0) 0x7f747c033020 con 0x55bfd50ad3b0 2024-04-25T10:59:30.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.572+0000 7f7499bc8740 10 client.4626 fetch_fsmap finished waiting for FSMap version 6 2024-04-25T10:59:30.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.572+0000 7f7499bc8740 10 client.4626 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:59:30.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.572+0000 7f7499bc8740 20 client.4626 populate_metadata read hostname 'smithi069' 2024-04-25T10:59:30.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.572+0000 7f7499bc8740 1 --2- 172.21.15.69:0/2677050469 >> [v2:172.21.15.146:6834/908008797,v1:172.21.15.146:6835/908008797] conn(0x55bfd5094e40 0x55bfd54a3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:30.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.572+0000 7f7499bc8740 4 client.4626 mds_command: new command op to 4294 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:59:30.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.572+0000 7f7499bc8740 1 -- 172.21.15.69:0/2677050469 --> [v2:172.21.15.146:6834/908008797,v1:172.21.15.146:6835/908008797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfd53ea350 con 0x55bfd5094e40 2024-04-25T10:59:30.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.572+0000 7f748bfff640 1 --2- 172.21.15.69:0/2677050469 >> [v2:172.21.15.146:6834/908008797,v1:172.21.15.146:6835/908008797] conn(0x55bfd5094e40 0x55bfd54a3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:30.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.573+0000 7f746b7fe640 10 client.4626 ms_handle_connect on v2:172.21.15.146:6834/908008797 2024-04-25T10:59:30.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.573+0000 7f748bfff640 1 --2- 172.21.15.69:0/2677050469 >> [v2:172.21.15.146:6834/908008797,v1:172.21.15.146:6835/908008797] conn(0x55bfd5094e40 0x55bfd54a3450 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=320b8ec26167d14 server_cookie=23bf413bb3aaf5aa in_seq=0 out_seq=0 2024-04-25T10:59:30.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.574+0000 7f746b7fe640 1 -- 172.21.15.69:0/2677050469 <== mds.0 v2:172.21.15.146:6834/908008797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bfd53ea350 con 0x55bfd5094e40 2024-04-25T10:59:30.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.574+0000 7f746b7fe640 10 client.4626 handle_command_reply: tid=0 2024-04-25T10:59:30.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.574+0000 7f7499bc8740 1 client.4626 shutdown 2024-04-25T10:59:30.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.574+0000 7f7499bc8740 1 -- 172.21.15.69:0/2677050469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd50ad3b0 msgr2=0x55bfd549b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:30.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.574+0000 7f7499bc8740 1 --2- 172.21.15.69:0/2677050469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd50ad3b0 0x55bfd549b1a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f747c0156c0 tx=0x7f747c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.574+0000 7f7499bc8740 1 -- 172.21.15.69:0/2677050469 shutdown_connections 2024-04-25T10:59:30.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.574+0000 7f7499bc8740 1 --2- 172.21.15.69:0/2677050469 >> [v2:172.21.15.146:6834/908008797,v1:172.21.15.146:6835/908008797] conn(0x55bfd5094e40 0x55bfd54a3450 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.574+0000 7f7499bc8740 1 --2- 172.21.15.69:0/2677050469 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfd50d6270 0x55bfd549e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.574+0000 7f7499bc8740 1 --2- 172.21.15.69:0/2677050469 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfd50b5590 0x55bfd54b41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.574+0000 7f746b7fe640 0 client.4626 ms_handle_reset on v2:172.21.15.146:6834/908008797 2024-04-25T10:59:30.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.574+0000 7f7499bc8740 1 --2- 172.21.15.69:0/2677050469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd50ad3b0 0x55bfd549b1a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.574+0000 7f7499bc8740 1 -- 172.21.15.69:0/2677050469 >> 172.21.15.69:0/2677050469 conn(0x55bfd50ca3c0 msgr2=0x55bfd5475fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:30.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.574+0000 7f7499bc8740 1 -- 172.21.15.69:0/2677050469 shutdown_connections 2024-04-25T10:59:30.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.574+0000 7f7499bc8740 1 -- 172.21.15.69:0/2677050469 wait complete. 2024-04-25T10:59:30.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.574+0000 7f7499bc8740 20 client.4626 trim_cache size 0 max 16384 2024-04-25T10:59:30.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.593+0000 7f7499bc8740 1 Processor -- start 2024-04-25T10:59:30.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.593+0000 7f7499bc8740 1 -- start start 2024-04-25T10:59:30.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.594+0000 7f7499bc8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd50ad3b0 0x55bfd54b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:30.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.594+0000 7f7499bc8740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfd50b5590 0x55bfd54bc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:30.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.594+0000 7f7499bc8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfd50d6270 0x55bfd54a1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:30.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.594+0000 7f7499bc8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f748c1548f0 con 0x55bfd50ad3b0 2024-04-25T10:59:30.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.594+0000 7f7499bc8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f748c153c50 con 0x55bfd50d6270 2024-04-25T10:59:30.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.594+0000 7f7499bc8740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f748c1542a0 con 0x55bfd50b5590 2024-04-25T10:59:30.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.594+0000 7f7490ab0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd50ad3b0 0x55bfd54b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:30.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.594+0000 7f7490ab0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd50ad3b0 0x55bfd54b41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56492/0 (socket says 172.21.15.69:56492) 2024-04-25T10:59:30.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.594+0000 7f7490ab0640 1 -- 172.21.15.69:0/1697743444 learned_addr learned my addr 172.21.15.69:0/1697743444 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:30.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.595+0000 7f7490ab0640 1 -- 172.21.15.69:0/1697743444 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfd50b5590 msgr2=0x55bfd54bc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:30.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.595+0000 7f7490ab0640 1 --2- 172.21.15.69:0/1697743444 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfd50b5590 0x55bfd54bc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.595+0000 7f7490ab0640 1 -- 172.21.15.69:0/1697743444 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfd50d6270 msgr2=0x55bfd54a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:30.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.595+0000 7f7490ab0640 1 --2- 172.21.15.69:0/1697743444 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfd50d6270 0x55bfd54a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.595+0000 7f7490ab0640 1 -- 172.21.15.69:0/1697743444 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfd53e19c0 con 0x55bfd50ad3b0 2024-04-25T10:59:30.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.595+0000 7f7490ab0640 1 --2- 172.21.15.69:0/1697743444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd50ad3b0 0x55bfd54b41b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f747c014b50 tx=0x7f747c021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b797dd2d4817012a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:30.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.595+0000 7f7471ffb640 1 -- 172.21.15.69:0/1697743444 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c038070 con 0x55bfd50ad3b0 2024-04-25T10:59:30.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.595+0000 7f7471ffb640 1 -- 172.21.15.69:0/1697743444 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f747c01a040 con 0x55bfd50ad3b0 2024-04-25T10:59:30.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.595+0000 7f7471ffb640 1 -- 172.21.15.69:0/1697743444 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c025070 con 0x55bfd50ad3b0 2024-04-25T10:59:30.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.596+0000 7f7499bc8740 1 -- 172.21.15.69:0/1697743444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd50ad3b0 msgr2=0x55bfd54b41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:30.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.596+0000 7f7499bc8740 1 --2- 172.21.15.69:0/1697743444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd50ad3b0 0x55bfd54b41b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f747c014b50 tx=0x7f747c021000 comp rx=0 tx=0).stop 2024-04-25T10:59:30.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.596+0000 7f7499bc8740 1 -- 172.21.15.69:0/1697743444 shutdown_connections 2024-04-25T10:59:30.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.596+0000 7f7499bc8740 1 --2- 172.21.15.69:0/1697743444 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfd50d6270 0x55bfd54a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.596+0000 7f7499bc8740 1 --2- 172.21.15.69:0/1697743444 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfd50b5590 0x55bfd54bc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.596+0000 7f7499bc8740 1 --2- 172.21.15.69:0/1697743444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd50ad3b0 0x55bfd54b41b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.596+0000 7f7499bc8740 1 -- 172.21.15.69:0/1697743444 >> 172.21.15.69:0/1697743444 conn(0x55bfd50ca3c0 msgr2=0x55bfd54a5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:30.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.596+0000 7f7499bc8740 1 -- 172.21.15.69:0/1697743444 shutdown_connections 2024-04-25T10:59:30.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.596+0000 7f7499bc8740 1 -- 172.21.15.69:0/1697743444 wait complete. 2024-04-25T10:59:30.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.597+0000 7f7499bc8740 1 -- start start 2024-04-25T10:59:30.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.597+0000 7f7499bc8740 1 Processor -- start 2024-04-25T10:59:30.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.598+0000 7f7499bc8740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfd50ad3b0 0x55bfd5539d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:30.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.598+0000 7f7499bc8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd50b5590 0x55bfd54a32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:30.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.598+0000 7f7499bc8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfd50d6270 0x55bfd54a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:30.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.598+0000 7f7499bc8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bfd549eda0 con 0x55bfd50b5590 2024-04-25T10:59:30.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.598+0000 7f7499bc8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55bfd54a5dd0 con 0x55bfd50d6270 2024-04-25T10:59:30.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.598+0000 7f7499bc8740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x55bfd54bbc30 con 0x55bfd50ad3b0 2024-04-25T10:59:30.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.598+0000 7f7499bc8740 10 client.0 _ll_register_callbacks cb 0x7f747c009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:59:30.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.598+0000 7f748bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd50b5590 0x55bfd54a32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:30.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.598+0000 7f748bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd50b5590 0x55bfd54a32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56502/0 (socket says 172.21.15.69:56502) 2024-04-25T10:59:30.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.598+0000 7f748bfff640 1 -- 172.21.15.69:0/3214599863 learned_addr learned my addr 172.21.15.69:0/3214599863 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:30.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.598+0000 7f748bfff640 1 -- 172.21.15.69:0/3214599863 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfd50ad3b0 msgr2=0x55bfd5539d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:30.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.598+0000 7f748bfff640 1 --2- 172.21.15.69:0/3214599863 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfd50ad3b0 0x55bfd5539d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.598+0000 7f748bfff640 1 -- 172.21.15.69:0/3214599863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfd50d6270 msgr2=0x55bfd54a37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:30.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.598+0000 7f748bfff640 1 --2- 172.21.15.69:0/3214599863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfd50d6270 0x55bfd54a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.598+0000 7f748bfff640 1 -- 172.21.15.69:0/3214599863 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfd54b9d20 con 0x55bfd50b5590 2024-04-25T10:59:30.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.599+0000 7f748bfff640 1 --2- 172.21.15.69:0/3214599863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd50b5590 0x55bfd54a32a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f748005b900 tx=0x7f7480007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1a5e0281baafd28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:30.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.599+0000 7f746b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-25T10:59:30.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.599+0000 7f746b7fe640 1 -- 172.21.15.69:0/3214599863 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748005daf0 con 0x55bfd50b5590 2024-04-25T10:59:30.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.599+0000 7f746b7fe640 1 -- 172.21.15.69:0/3214599863 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7480073ae0 con 0x55bfd50b5590 2024-04-25T10:59:30.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.599+0000 7f7499bc8740 1 -- 172.21.15.69:0/3214599863 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f747c01a040 con 0x55bfd50b5590 2024-04-25T10:59:30.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.599+0000 7f746b7fe640 1 -- 172.21.15.69:0/3214599863 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7f748005e690 con 0x55bfd50b5590 2024-04-25T10:59:30.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.599+0000 7f7499bc8740 10 client.4632 fetch_fsmap learned FSMap version 6 2024-04-25T10:59:30.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.599+0000 7f7499bc8740 1 -- 172.21.15.69:0/3214599863 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55bfd549e300 con 0x55bfd50b5590 2024-04-25T10:59:30.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.599+0000 7f746b7fe640 1 -- 172.21.15.69:0/3214599863 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f747c01a040 con 0x55bfd50b5590 2024-04-25T10:59:30.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.599+0000 7f746b7fe640 1 -- 172.21.15.69:0/3214599863 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 6) ==== 3494+0+0 (secure 0 0 0) 0x7f7480073ce0 con 0x55bfd50b5590 2024-04-25T10:59:30.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.599+0000 7f7499bc8740 10 client.4632 fetch_fsmap finished waiting for FSMap version 6 2024-04-25T10:59:30.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.599+0000 7f7499bc8740 10 client.4632 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:59:30.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.599+0000 7f7499bc8740 20 client.4632 populate_metadata read hostname 'smithi069' 2024-04-25T10:59:30.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.599+0000 7f7499bc8740 1 --2- 172.21.15.69:0/3214599863 >> [v2:172.21.15.146:6834/908008797,v1:172.21.15.146:6835/908008797] conn(0x55bfd54bc920 0x55bfd54be890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:30.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.599+0000 7f7499bc8740 4 client.4632 mds_command: new command op to 4294 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:59:30.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.599+0000 7f7499bc8740 1 -- 172.21.15.69:0/3214599863 --> [v2:172.21.15.146:6834/908008797,v1:172.21.15.146:6835/908008797] -- command(tid 0: {"prefix": "status"}) -- 0x55bfd53e41c0 con 0x55bfd54bc920 2024-04-25T10:59:30.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.599+0000 7f7490ab0640 1 --2- 172.21.15.69:0/3214599863 >> [v2:172.21.15.146:6834/908008797,v1:172.21.15.146:6835/908008797] conn(0x55bfd54bc920 0x55bfd54be890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:30.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.600+0000 7f746b7fe640 10 client.4632 ms_handle_connect on v2:172.21.15.146:6834/908008797 2024-04-25T10:59:30.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.600+0000 7f7490ab0640 1 --2- 172.21.15.69:0/3214599863 >> [v2:172.21.15.146:6834/908008797,v1:172.21.15.146:6835/908008797] conn(0x55bfd54bc920 0x55bfd54be890 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=617399e2532acd88 server_cookie=7a352ee78a61cb04 in_seq=0 out_seq=0 2024-04-25T10:59:30.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.600+0000 7f746b7fe640 1 -- 172.21.15.69:0/3214599863 <== mds.0 v2:172.21.15.146:6834/908008797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55bfd53e41c0 con 0x55bfd54bc920 2024-04-25T10:59:30.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.600+0000 7f746b7fe640 10 client.4632 handle_command_reply: tid=0 2024-04-25T10:59:30.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.600+0000 7f7499bc8740 1 client.4632 shutdown 2024-04-25T10:59:30.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.600+0000 7f7499bc8740 1 -- 172.21.15.69:0/3214599863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd50b5590 msgr2=0x55bfd54a32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:30.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.600+0000 7f7499bc8740 1 --2- 172.21.15.69:0/3214599863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd50b5590 0x55bfd54a32a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f748005b900 tx=0x7f7480007780 comp rx=0 tx=0).stop 2024-04-25T10:59:30.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.600+0000 7f7499bc8740 1 -- 172.21.15.69:0/3214599863 shutdown_connections 2024-04-25T10:59:30.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.600+0000 7f7499bc8740 1 --2- 172.21.15.69:0/3214599863 >> [v2:172.21.15.146:6834/908008797,v1:172.21.15.146:6835/908008797] conn(0x55bfd54bc920 0x55bfd54be890 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.600+0000 7f7499bc8740 1 --2- 172.21.15.69:0/3214599863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfd50d6270 0x55bfd54a37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.600+0000 7f7499bc8740 1 --2- 172.21.15.69:0/3214599863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd50b5590 0x55bfd54a32a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.600+0000 7f746b7fe640 0 client.4632 ms_handle_reset on v2:172.21.15.146:6834/908008797 2024-04-25T10:59:30.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.600+0000 7f7499bc8740 1 --2- 172.21.15.69:0/3214599863 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfd50ad3b0 0x55bfd5539d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.600+0000 7f7499bc8740 1 -- 172.21.15.69:0/3214599863 >> 172.21.15.69:0/3214599863 conn(0x55bfd50ca3c0 msgr2=0x55bfd549a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:30.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.600+0000 7f7499bc8740 1 -- 172.21.15.69:0/3214599863 shutdown_connections 2024-04-25T10:59:30.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.600+0000 7f7499bc8740 1 -- 172.21.15.69:0/3214599863 wait complete. 2024-04-25T10:59:30.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.600+0000 7f7499bc8740 20 client.4632 trim_cache size 0 max 16384 2024-04-25T10:59:30.608 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T10:59:30.608 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "ed0c8fe4-1bbb-4705-be72-eeea5bef859e", 2024-04-25T10:59:30.608 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 0, 2024-04-25T10:59:30.608 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 4294, 2024-04-25T10:59:30.608 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-04-25T10:59:30.609 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-04-25T10:59:30.609 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-04-25T10:59:30.609 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 10.982876904999999, 2024-04-25T10:59:30.609 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 6, 2024-04-25T10:59:30.609 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 20, 2024-04-25T10:59:30.609 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 20, 2024-04-25T10:59:30.609 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 18.358848656999999 2024-04-25T10:59:30.609 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T10:59:30.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.601+0000 7f7492d14640 1 -- 172.21.15.69:0/1440922993 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f74780744b0 msgr2=0x7f74780769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:30.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.601+0000 7f7492d14640 1 --2- 172.21.15.69:0/1440922993 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f74780744b0 0x7f74780769c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f747c009a30 tx=0x7f747c008040 comp rx=0 tx=0).stop 2024-04-25T10:59:30.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.601+0000 7f7492d14640 1 -- 172.21.15.69:0/1440922993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f748c160fa0 msgr2=0x7f748c1555d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:30.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.601+0000 7f7492d14640 1 --2- 172.21.15.69:0/1440922993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f748c160fa0 0x7f748c1555d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f748000c3f0 tx=0x7f748000c8b0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.601+0000 7f7492d14640 1 -- 172.21.15.69:0/1440922993 shutdown_connections 2024-04-25T10:59:30.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.601+0000 7f7492d14640 1 --2- 172.21.15.69:0/1440922993 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f74780744b0 0x7f74780769c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.601+0000 7f7492d14640 1 --2- 172.21.15.69:0/1440922993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f748c164b00 0x7f748c155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.601+0000 7f7492d14640 1 --2- 172.21.15.69:0/1440922993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f748c160fa0 0x7f748c1555d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.601+0000 7f7492d14640 1 --2- 172.21.15.69:0/1440922993 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f748c15df80 0x7f748c1521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:30.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.601+0000 7f7492d14640 1 -- 172.21.15.69:0/1440922993 >> 172.21.15.69:0/1440922993 conn(0x7f748c0d2bc0 msgr2=0x7f748c15bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:30.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.601+0000 7f7492d14640 1 -- 172.21.15.69:0/1440922993 shutdown_connections 2024-04-25T10:59:30.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:30.601+0000 7f7492d14640 1 -- 172.21.15.69:0/1440922993 wait complete. 2024-04-25T10:59:30.613 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T10:59:30.625 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T10:59:30.625 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T10:59:30.626 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T10:59:30.626 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T10:59:30.690 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T10:59:30.690 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-25T10:59:30.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:30.691 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2024-04-25T10:59:30.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:30.707 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link delete ceph-brx 2024-04-25T10:59:30.771 INFO:teuthology.orchestra.run.smithi069.stderr:Cannot find device "ceph-brx" 2024-04-25T10:59:30.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:59:30.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:30.773 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2024-04-25T10:59:30.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:30.829 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link delete ceph-brx 2024-04-25T10:59:30.893 INFO:teuthology.orchestra.run.smithi069.stderr:Cannot find device "ceph-brx" 2024-04-25T10:59:30.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:59:30.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:30.895 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-04-25T10:59:30.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:30.914 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link delete ceph-brx 2024-04-25T10:59:30.985 INFO:teuthology.orchestra.run.smithi146.stderr:Cannot find device "ceph-brx" 2024-04-25T10:59:30.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:59:30.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:30.987 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-04-25T10:59:31.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:31.045 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link delete ceph-brx 2024-04-25T10:59:31.114 INFO:teuthology.orchestra.run.smithi146.stderr:Cannot find device "ceph-brx" 2024-04-25T10:59:31.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:59:31.116 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-04-25T10:59:31.116 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:59:31.116 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:59:31.117 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:59:31.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.224+0000 7f9ca79e3640 1 Processor -- start 2024-04-25T10:59:31.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.224+0000 7f9ca79e3640 1 -- start start 2024-04-25T10:59:31.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.225+0000 7f9ca79e3640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ca00cf770 0x7f9ca0162590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:31.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.225+0000 7f9ca79e3640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9ca0162ad0 0x7f9ca00b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:31.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.225+0000 7f9ca79e3640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ca00cfc30 con 0x7f9ca00b91d0 2024-04-25T10:59:31.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.225+0000 7f9ca79e3640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f9ca00d1ff0 con 0x7f9ca0162ad0 2024-04-25T10:59:31.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.225+0000 7f9ca79e3640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9ca00d2890 con 0x7f9ca00cf770 2024-04-25T10:59:31.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.225+0000 7f9ca577f640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f9ca00b91d0 0x7f9ca0161e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:57274/0 (socket says 172.21.15.69:57274) 2024-04-25T10:59:31.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.225+0000 7f9ca577f640 1 -- 172.21.15.69:0/3760107881 learned_addr learned my addr 172.21.15.69:0/3760107881 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:31.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.226+0000 7f9c97fff640 1 -- 172.21.15.69:0/3760107881 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4218144381 0 0) 0x7f9ca00cfc30 con 0x7f9ca00b91d0 2024-04-25T10:59:31.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.226+0000 7f9c97fff640 1 -- 172.21.15.69:0/3760107881 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c8c003520 con 0x7f9ca00b91d0 2024-04-25T10:59:31.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.226+0000 7f9ca5f80640 1 --2- 172.21.15.69:0/3760107881 >> v2:172.21.15.146:3300/0 conn(0x7f9ca0162ad0 0x7f9ca00b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:31.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.226+0000 7f9ca4f7e640 1 --2- 172.21.15.69:0/3760107881 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ca00cf770 0x7f9ca0162590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:31.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.226+0000 7f9c97fff640 1 -- 172.21.15.69:0/3760107881 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3570895990 0 0) 0x7f9c8c003520 con 0x7f9ca00b91d0 2024-04-25T10:59:31.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.226+0000 7f9c97fff640 1 -- 172.21.15.69:0/3760107881 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ca00cf770 msgr2=0x7f9ca0162590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:31.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.226+0000 7f9c97fff640 1 --2- 172.21.15.69:0/3760107881 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ca00cf770 0x7f9ca0162590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:31.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.226+0000 7f9c97fff640 1 -- 172.21.15.69:0/3760107881 >> v2:172.21.15.146:3300/0 conn(0x7f9ca0162ad0 msgr2=0x7f9ca00b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:31.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.226+0000 7f9c97fff640 1 --2- 172.21.15.69:0/3760107881 >> v2:172.21.15.146:3300/0 conn(0x7f9ca0162ad0 0x7f9ca00b0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:31.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.226+0000 7f9c97fff640 1 -- 172.21.15.69:0/3760107881 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca00d2240 con 0x7f9ca00b91d0 2024-04-25T10:59:31.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.226+0000 7f9c97fff640 1 -- 172.21.15.69:0/3760107881 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7f9c9c001670 con 0x7f9ca00b91d0 2024-04-25T10:59:31.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.226+0000 7f9ca4f7e640 1 --2- 172.21.15.69:0/3760107881 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ca00cf770 0x7f9ca0162590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:31.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.226+0000 7f9c97fff640 1 -- 172.21.15.69:0/3760107881 >> v1:172.21.15.69:6789/0 conn(0x7f9ca00b91d0 legacy=0x7f9ca0161e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:31.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.226+0000 7f9c97fff640 1 --2- 172.21.15.69:0/3760107881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c8c0039b0 0x7f9c8c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:31.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.226+0000 7f9c97fff640 1 --2- 172.21.15.69:0/3760107881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c8c004800 0x7f9c8c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:31.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.226+0000 7f9c97fff640 1 --2- 172.21.15.69:0/3760107881 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c8c005150 0x7f9c8c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:31.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.226+0000 7f9c97fff640 1 -- 172.21.15.69:0/3760107881 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca00d1ff0 con 0x7f9c8c004800 2024-04-25T10:59:31.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.226+0000 7f9c97fff640 1 -- 172.21.15.69:0/3760107881 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca00d2890 con 0x7f9c8c0039b0 2024-04-25T10:59:31.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.226+0000 7f9c97fff640 1 -- 172.21.15.69:0/3760107881 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9c8c00bf00 con 0x7f9c8c005150 2024-04-25T10:59:31.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.226+0000 7f9ca4f7e640 1 --2- 172.21.15.69:0/3760107881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c8c004800 0x7f9c8c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:31.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.226+0000 7f9ca577f640 1 --2- 172.21.15.69:0/3760107881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c8c0039b0 0x7f9c8c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:31.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.226+0000 7f9ca5f80640 1 --2- 172.21.15.69:0/3760107881 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c8c005150 0x7f9c8c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:31.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.226+0000 7f9ca4f7e640 1 -- 172.21.15.69:0/3760107881 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c8c005150 msgr2=0x7f9c8c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:31.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.226+0000 7f9ca4f7e640 1 --2- 172.21.15.69:0/3760107881 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c8c005150 0x7f9c8c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:31.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.226+0000 7f9ca4f7e640 1 -- 172.21.15.69:0/3760107881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c8c0039b0 msgr2=0x7f9c8c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:31.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.226+0000 7f9ca4f7e640 1 --2- 172.21.15.69:0/3760107881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c8c0039b0 0x7f9c8c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:31.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.226+0000 7f9ca4f7e640 1 -- 172.21.15.69:0/3760107881 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8c003520 con 0x7f9c8c004800 2024-04-25T10:59:31.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.227+0000 7f9ca4f7e640 1 --2- 172.21.15.69:0/3760107881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c8c004800 0x7f9c8c004c10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9c900098d0 tx=0x7f9c8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97698bcec7128fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:31.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.227+0000 7f9c97fff640 1 -- 172.21.15.69:0/3760107881 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9000f040 con 0x7f9c8c004800 2024-04-25T10:59:31.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.227+0000 7f9c97fff640 1 -- 172.21.15.69:0/3760107881 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c9002ad00 con 0x7f9c8c004800 2024-04-25T10:59:31.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.227+0000 7f9c97fff640 1 -- 172.21.15.69:0/3760107881 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90004250 con 0x7f9c8c004800 2024-04-25T10:59:31.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.228+0000 7f9ca79e3640 1 -- 172.21.15.69:0/3760107881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c8c004800 msgr2=0x7f9c8c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:31.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.228+0000 7f9ca79e3640 1 --2- 172.21.15.69:0/3760107881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c8c004800 0x7f9c8c004c10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9c900098d0 tx=0x7f9c8c005830 comp rx=0 tx=0).stop 2024-04-25T10:59:31.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.228+0000 7f9ca5f80640 1 -- 172.21.15.69:0/3760107881 reap_dead start 2024-04-25T10:59:31.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.228+0000 7f9ca79e3640 1 -- 172.21.15.69:0/3760107881 shutdown_connections 2024-04-25T10:59:31.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.228+0000 7f9ca79e3640 1 -- 172.21.15.69:0/3760107881 >> 172.21.15.69:0/3760107881 conn(0x7f9ca00d3aa0 msgr2=0x7f9ca00d3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:31.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.228+0000 7f9ca79e3640 1 -- 172.21.15.69:0/3760107881 shutdown_connections 2024-04-25T10:59:31.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.228+0000 7f9ca79e3640 1 -- 172.21.15.69:0/3760107881 wait complete. 2024-04-25T10:59:31.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.230+0000 7f9ca79e3640 1 Processor -- start 2024-04-25T10:59:31.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.230+0000 7f9ca79e3640 1 -- start start 2024-04-25T10:59:31.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.231+0000 7f9ca79e3640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c8c004800 0x7f9ca01f69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:31.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.231+0000 7f9ca79e3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca006e6e0 0x7f9ca015f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:31.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.231+0000 7f9ca79e3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ca00b6d60 0x7f9ca00b7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:31.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.231+0000 7f9ca79e3640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca00b24a0 con 0x7f9ca006e6e0 2024-04-25T10:59:31.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.231+0000 7f9ca79e3640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca00b2320 con 0x7f9ca00b6d60 2024-04-25T10:59:31.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.231+0000 7f9ca79e3640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9ca00cf770 con 0x7f9c8c004800 2024-04-25T10:59:31.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.231+0000 7f9ca4f7e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca006e6e0 0x7f9ca015f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:31.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.231+0000 7f9ca4f7e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca006e6e0 0x7f9ca015f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56516/0 (socket says 172.21.15.69:56516) 2024-04-25T10:59:31.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.231+0000 7f9ca4f7e640 1 -- 172.21.15.69:0/3531048719 learned_addr learned my addr 172.21.15.69:0/3531048719 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:31.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.231+0000 7f9ca577f640 1 --2- 172.21.15.69:0/3531048719 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c8c004800 0x7f9ca01f69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:31.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.231+0000 7f9ca5f80640 1 --2- 172.21.15.69:0/3531048719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ca00b6d60 0x7f9ca00b7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:31.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.231+0000 7f9ca4f7e640 1 -- 172.21.15.69:0/3531048719 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c8c004800 msgr2=0x7f9ca01f69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:31.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.231+0000 7f9ca4f7e640 1 --2- 172.21.15.69:0/3531048719 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c8c004800 0x7f9ca01f69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:31.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.231+0000 7f9ca4f7e640 1 -- 172.21.15.69:0/3531048719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ca00b6d60 msgr2=0x7f9ca00b7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:31.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.231+0000 7f9ca4f7e640 1 --2- 172.21.15.69:0/3531048719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ca00b6d60 0x7f9ca00b7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:31.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.231+0000 7f9ca4f7e640 1 -- 172.21.15.69:0/3531048719 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca00b76b0 con 0x7f9ca006e6e0 2024-04-25T10:59:31.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.231+0000 7f9ca4f7e640 1 --2- 172.21.15.69:0/3531048719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca006e6e0 0x7f9ca015f710 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9c90006fd0 tx=0x7f9c90004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:31.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.232+0000 7f9c95ffb640 1 -- 172.21.15.69:0/3531048719 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90041070 con 0x7f9ca006e6e0 2024-04-25T10:59:31.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.232+0000 7f9c95ffb640 1 -- 172.21.15.69:0/3531048719 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c9000f000 con 0x7f9ca006e6e0 2024-04-25T10:59:31.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.232+0000 7f9c95ffb640 1 -- 172.21.15.69:0/3531048719 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90003bd0 con 0x7f9ca006e6e0 2024-04-25T10:59:31.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.232+0000 7f9ca79e3640 1 -- 172.21.15.69:0/3531048719 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ca006d920 con 0x7f9ca006e6e0 2024-04-25T10:59:31.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.232+0000 7f9ca79e3640 1 -- 172.21.15.69:0/3531048719 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ca006dee0 con 0x7f9ca006e6e0 2024-04-25T10:59:31.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.233+0000 7f9ca79e3640 1 -- 172.21.15.69:0/3531048719 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ca0050740 con 0x7f9ca006e6e0 2024-04-25T10:59:31.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.233+0000 7f9c95ffb640 1 -- 172.21.15.69:0/3531048719 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9c90044070 con 0x7f9ca006e6e0 2024-04-25T10:59:31.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.234+0000 7f9c95ffb640 1 --2- 172.21.15.69:0/3531048719 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f9c840744e0 0x7f9c840769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:31.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.234+0000 7f9c95ffb640 1 -- 172.21.15.69:0/3531048719 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9c900b5a80 con 0x7f9ca006e6e0 2024-04-25T10:59:31.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.234+0000 7f9ca577f640 1 --2- 172.21.15.69:0/3531048719 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f9c840744e0 0x7f9c840769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:31.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.234+0000 7f9ca577f640 1 --2- 172.21.15.69:0/3531048719 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f9c840744e0 0x7f9c840769f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c001470 tx=0x7f9c9c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:31.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.238+0000 7f9c95ffb640 1 -- 172.21.15.69:0/3531048719 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9ca00b76b0 con 0x7f9ca006e6e0 2024-04-25T10:59:31.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.451+0000 7f9ca79e3640 1 -- 172.21.15.69:0/3531048719 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9ca006f5b0 con 0x7f9ca006e6e0 2024-04-25T10:59:31.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.452+0000 7f9c95ffb640 1 -- 172.21.15.69:0/3531048719 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7f9c90080210 con 0x7f9ca006e6e0 2024-04-25T10:59:31.452 INFO:teuthology.orchestra.run.smithi069.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:59:31.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.454+0000 7f9ca79e3640 1 -- 172.21.15.69:0/3531048719 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f9c840744e0 msgr2=0x7f9c840769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:31.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.454+0000 7f9ca79e3640 1 --2- 172.21.15.69:0/3531048719 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f9c840744e0 0x7f9c840769f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c001470 tx=0x7f9c9c009040 comp rx=0 tx=0).stop 2024-04-25T10:59:31.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.454+0000 7f9ca79e3640 1 -- 172.21.15.69:0/3531048719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca006e6e0 msgr2=0x7f9ca015f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:31.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.454+0000 7f9ca79e3640 1 --2- 172.21.15.69:0/3531048719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca006e6e0 0x7f9ca015f710 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9c90006fd0 tx=0x7f9c90004730 comp rx=0 tx=0).stop 2024-04-25T10:59:31.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.454+0000 7f9ca79e3640 1 -- 172.21.15.69:0/3531048719 shutdown_connections 2024-04-25T10:59:31.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.454+0000 7f9ca79e3640 1 --2- 172.21.15.69:0/3531048719 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f9c840744e0 0x7f9c840769f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:31.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.454+0000 7f9ca79e3640 1 --2- 172.21.15.69:0/3531048719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ca00b6d60 0x7f9ca00b7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:31.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.454+0000 7f9ca79e3640 1 --2- 172.21.15.69:0/3531048719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca006e6e0 0x7f9ca015f710 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:31.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.454+0000 7f9ca79e3640 1 --2- 172.21.15.69:0/3531048719 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c8c004800 0x7f9ca01f69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:31.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.454+0000 7f9ca79e3640 1 -- 172.21.15.69:0/3531048719 >> 172.21.15.69:0/3531048719 conn(0x7f9ca00d3aa0 msgr2=0x7f9ca00d01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:31.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.454+0000 7f9ca79e3640 1 -- 172.21.15.69:0/3531048719 shutdown_connections 2024-04-25T10:59:31.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31.454+0000 7f9ca79e3640 1 -- 172.21.15.69:0/3531048719 wait complete. 2024-04-25T10:59:31.466 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:59:31.466 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:59:31.466 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi069.front.sepia.ceph.com 2024-04-25T10:59:31.466 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:59:31.466 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:59:31.466 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:59:31.466 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:59:31.466 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:59:31.466 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:59:31.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:31.467 DEBUG:teuthology.orchestra.run.smithi069:> ip addr 2024-04-25T10:59:31.482 INFO:teuthology.orchestra.run.smithi069.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:59:31.482 INFO:teuthology.orchestra.run.smithi069.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:59:31.482 INFO:teuthology.orchestra.run.smithi069.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:59:31.482 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:31.482 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 ::1/128 scope host 2024-04-25T10:59:31.482 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:31.482 INFO:teuthology.orchestra.run.smithi069.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:59:31.483 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:d8:d9:c2 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:59:31.483 INFO:teuthology.orchestra.run.smithi069.stdout: altname enp7s0f0 2024-04-25T10:59:31.483 INFO:teuthology.orchestra.run.smithi069.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:59:31.483 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:d8:d9:c3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:59:31.483 INFO:teuthology.orchestra.run.smithi069.stdout: altname enp7s0f1 2024-04-25T10:59:31.483 INFO:teuthology.orchestra.run.smithi069.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:59:31.483 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:8f:d7:36 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:59:31.483 INFO:teuthology.orchestra.run.smithi069.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:59:31.483 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:8f:d7:37 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:59:31.483 INFO:teuthology.orchestra.run.smithi069.stdout: inet 172.21.15.69/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T10:59:31.483 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft 42745sec preferred_lft 42745sec 2024-04-25T10:59:31.483 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 fe80::adbc:4e47:e7f9:8987/64 scope link noprefixroute 2024-04-25T10:59:31.483 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:31.484 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T10:59:31.484 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:59:31.484 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-04-25T10:59:31.484 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link add name ceph-brx type bridge 2024-04-25T10:59:31.485 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip addr flush dev ceph-brx 2024-04-25T10:59:31.485 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set ceph-brx up 2024-04-25T10:59:31.485 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T10:59:31.485 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-04-25T10:59:31.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:59:31.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:31.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:31.684 DEBUG:teuthology.orchestra.run.smithi069:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:59:31.706 INFO:teuthology.orchestra.run.smithi069.stdout:1 2024-04-25T10:59:31.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:31.707 DEBUG:teuthology.orchestra.run.smithi069:> route 2024-04-25T10:59:31.771 INFO:teuthology.orchestra.run.smithi069.stdout:Kernel IP routing table 2024-04-25T10:59:31.772 INFO:teuthology.orchestra.run.smithi069.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:59:31.772 INFO:teuthology.orchestra.run.smithi069.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:59:31.772 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:59:31.772 INFO:teuthology.orchestra.run.smithi069.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:59:31.772 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:59:31.773 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-04-25T10:59:31.773 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:59:31.773 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:59:31.773 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:59:31.773 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-04-25T10:59:31.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:59:32.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:32.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:32.028 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2024-04-25T10:59:32.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:32.043 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list-id 2024-04-25T10:59:32.100 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:59:32.100 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-04-25T10:59:32.101 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:59:32.101 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T10:59:32.101 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-04-25T10:59:32.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:59:32.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:32.215 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T10:59:32.215 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:59:32.216 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-04-25T10:59:32.216 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T10:59:32.216 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T10:59:32.216 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T10:59:32.216 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T10:59:32.216 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T10:59:32.216 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-04-25T10:59:32.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:59:32.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:32.331 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:59:32.331 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-04-25T10:59:32.331 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set brx.0 up 2024-04-25T10:59:32.331 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T10:59:32.331 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-04-25T10:59:32.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:59:32.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:32.405 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:59:32.405 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:59:32.419 INFO:teuthology.orchestra.run.smithi069.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:59:32.419 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:59:32.420 DEBUG:teuthology.orchestra.run.smithi069:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:59:32.473 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T10:59:32.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:32.474 DEBUG:teuthology.orchestra.run.smithi069:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:59:32.583 INFO:teuthology.orchestra.run.smithi069.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:59:32.583 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:59:32.698 INFO:teuthology.orchestra.run.smithi069.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:59:32.698 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:59:32.698 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: resolved to: "172.21.15.69:3300,172.21.15.146:3300,172.21.15.146:3301" 2024-04-25T10:59:32.698 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: trying mount with new device syntax: 0@ed0c8fe4-1bbb-4705-be72-eeea5bef859e.cephfs=/ 2024-04-25T10:59:32.699 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.69:3300/172.21.15.146:3300/172.21.15.146:3301" will pass to kernel 2024-04-25T10:59:32.699 INFO:teuthology.orchestra.run.smithi069.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:59:32.699 INFO:teuthology.orchestra.run.smithi069.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:59:32.699 INFO:teuthology.orchestra.run.smithi069.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:59:32.699 INFO:teuthology.orchestra.run.smithi069.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:59:32.699 INFO:teuthology.orchestra.run.smithi069.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:59:32.701 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:59:32.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:32.701 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:59:32.725 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:59:32.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T10:59:32.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:32.828 INFO:teuthology.orchestra.run.smithi069.stdout:client4567 2024-04-25T10:59:32.828 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:59:32.828 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:59:32.844 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:59:32.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T10:59:32.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:32.947 INFO:teuthology.orchestra.run.smithi069.stdout:client4567 2024-04-25T10:59:32.947 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:59:32.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T10:59:33.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:33.009 INFO:teuthology.orchestra.run.smithi069.stdout:client4567 2024-04-25T10:59:33.009 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/ed0c8fe4-1bbb-4705-be72-eeea5bef859e.client4567/status') 2024-04-25T10:59:33.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:59:33.072 INFO:teuthology.orchestra.run.smithi069.stdout:instance: client.4567 (3)172.21.15.69:0/1242200260 2024-04-25T10:59:33.072 INFO:teuthology.orchestra.run.smithi069.stdout:blocklisted: false 2024-04-25T10:59:33.073 INFO:teuthology.orchestra.run.smithi069.stderr:0+1 records in 2024-04-25T10:59:33.073 INFO:teuthology.orchestra.run.smithi069.stderr:0+1 records out 2024-04-25T10:59:33.073 INFO:teuthology.orchestra.run.smithi069.stderr:70 bytes copied, 2.1935e-05 s, 3.2 MB/s 2024-04-25T10:59:33.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:33.080 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-04-25T10:59:33.080 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:59:33.080 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T10:59:33.080 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:59:33.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.187+0000 7f22dab8f640 1 Processor -- start 2024-04-25T10:59:33.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.187+0000 7f22dab8f640 1 -- start start 2024-04-25T10:59:33.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.188+0000 7f22dab8f640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f22d40ce520 0x7f22d406c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:33.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.189+0000 7f22dab8f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22d40b7f50 0x7f22d406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:33.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.189+0000 7f22dab8f640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22d40ce930 con 0x7f22d415e320 2024-04-25T10:59:33.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.189+0000 7f22dab8f640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f22d40d0450 con 0x7f22d40ce520 2024-04-25T10:59:33.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.189+0000 7f22dab8f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f22d406cc50 con 0x7f22d40b7f50 2024-04-25T10:59:33.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.189+0000 7f22d912c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22d40b7f50 0x7f22d406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:33.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.189+0000 7f22d892b640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f22d40ce520 0x7f22d406c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:33.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.189+0000 7f22d892b640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f22d40ce520 0x7f22d406c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.69:39050/0 (socket says 172.21.15.69:39050) 2024-04-25T10:59:33.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.189+0000 7f22d912c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22d40b7f50 0x7f22d406dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.69:42032/0 (socket says 172.21.15.69:42032) 2024-04-25T10:59:33.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.189+0000 7f22d892b640 1 -- 172.21.15.69:0/3571506684 learned_addr learned my addr 172.21.15.69:0/3571506684 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:33.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.189+0000 7f22d3fff640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f22d415e320 0x7f22d40b7840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.69:44062/0 (socket says 172.21.15.69:44062) 2024-04-25T10:59:33.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.189+0000 7f22d892b640 1 -- 172.21.15.69:0/3571506684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22d40b7f50 msgr2=0x7f22d406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:33.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.189+0000 7f22d892b640 1 --2- 172.21.15.69:0/3571506684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22d40b7f50 0x7f22d406dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:33.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.189+0000 7f22d892b640 1 -- 172.21.15.69:0/3571506684 >> v1:172.21.15.146:6789/0 conn(0x7f22d415e320 legacy=0x7f22d40b7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:33.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.189+0000 7f22d892b640 1 -- 172.21.15.69:0/3571506684 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d406cf30 con 0x7f22d40ce520 2024-04-25T10:59:33.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.189+0000 7f22d37fe640 1 -- 172.21.15.69:0/3571506684 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1842401296 0 0) 0x7f22d40ce930 con 0x7f22d415e320 2024-04-25T10:59:33.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.189+0000 7f22d912c640 1 --2- 172.21.15.69:0/3571506684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22d40b7f50 0x7f22d406dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:59:33.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.190+0000 7f22d892b640 1 --2- 172.21.15.69:0/3571506684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22d40ce520 0x7f22d406c5c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f22c4009870 tx=0x7f22c402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f41435dce58f1a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:33.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.190+0000 7f22d37fe640 1 -- 172.21.15.69:0/3571506684 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c4037070 con 0x7f22d40ce520 2024-04-25T10:59:33.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.190+0000 7f22d37fe640 1 -- 172.21.15.69:0/3571506684 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22c4032440 con 0x7f22d40ce520 2024-04-25T10:59:33.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.190+0000 7f22d37fe640 1 -- 172.21.15.69:0/3571506684 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c4032810 con 0x7f22d40ce520 2024-04-25T10:59:33.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.191+0000 7f22dab8f640 1 -- 172.21.15.69:0/3571506684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22d40ce520 msgr2=0x7f22d406c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:33.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.191+0000 7f22dab8f640 1 --2- 172.21.15.69:0/3571506684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22d40ce520 0x7f22d406c5c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f22c4009870 tx=0x7f22c402a1c0 comp rx=0 tx=0).stop 2024-04-25T10:59:33.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.191+0000 7f22dab8f640 1 -- 172.21.15.69:0/3571506684 shutdown_connections 2024-04-25T10:59:33.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.191+0000 7f22dab8f640 1 --2- 172.21.15.69:0/3571506684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22d40b7f50 0x7f22d406dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:33.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.191+0000 7f22dab8f640 1 --2- 172.21.15.69:0/3571506684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22d40ce520 0x7f22d406c5c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:33.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.191+0000 7f22dab8f640 1 -- 172.21.15.69:0/3571506684 >> 172.21.15.69:0/3571506684 conn(0x7f22d409b7d0 msgr2=0x7f22d409bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:33.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.191+0000 7f22dab8f640 1 -- 172.21.15.69:0/3571506684 shutdown_connections 2024-04-25T10:59:33.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.191+0000 7f22dab8f640 1 -- 172.21.15.69:0/3571506684 wait complete. 2024-04-25T10:59:33.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.193+0000 7f22dab8f640 1 Processor -- start 2024-04-25T10:59:33.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.193+0000 7f22dab8f640 1 -- start start 2024-04-25T10:59:33.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.194+0000 7f22dab8f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22d40ce520 0x7f22d40c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:33.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.194+0000 7f22dab8f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22d415e320 0x7f22d414fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:33.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.194+0000 7f22dab8f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22d41f40d0 0x7f22d40c7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:33.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.194+0000 7f22dab8f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f22d406d7d0 con 0x7f22d415e320 2024-04-25T10:59:33.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.194+0000 7f22dab8f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f22d406d4f0 con 0x7f22d41f40d0 2024-04-25T10:59:33.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.194+0000 7f22dab8f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f22d415e730 con 0x7f22d40ce520 2024-04-25T10:59:33.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.194+0000 7f22d3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22d415e320 0x7f22d414fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:33.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.194+0000 7f22d3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22d415e320 0x7f22d414fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56522/0 (socket says 172.21.15.69:56522) 2024-04-25T10:59:33.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.194+0000 7f22d3fff640 1 -- 172.21.15.69:0/851042340 learned_addr learned my addr 172.21.15.69:0/851042340 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:33.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.194+0000 7f22d892b640 1 --2- 172.21.15.69:0/851042340 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22d40ce520 0x7f22d40c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:33.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.194+0000 7f22d912c640 1 --2- 172.21.15.69:0/851042340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22d41f40d0 0x7f22d40c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:33.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.195+0000 7f22d3fff640 1 -- 172.21.15.69:0/851042340 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22d40ce520 msgr2=0x7f22d40c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:33.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.195+0000 7f22d3fff640 1 --2- 172.21.15.69:0/851042340 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22d40ce520 0x7f22d40c9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:33.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.195+0000 7f22d3fff640 1 -- 172.21.15.69:0/851042340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22d41f40d0 msgr2=0x7f22d40c7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:33.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.195+0000 7f22d3fff640 1 --2- 172.21.15.69:0/851042340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22d41f40d0 0x7f22d40c7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:33.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.195+0000 7f22d3fff640 1 -- 172.21.15.69:0/851042340 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d40cb530 con 0x7f22d415e320 2024-04-25T10:59:33.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.195+0000 7f22d3fff640 1 --2- 172.21.15.69:0/851042340 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22d415e320 0x7f22d414fe00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f22c800c460 tx=0x7f22c800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:33.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.195+0000 7f22d17fa640 1 -- 172.21.15.69:0/851042340 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c80074e0 con 0x7f22d415e320 2024-04-25T10:59:33.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.195+0000 7f22d17fa640 1 -- 172.21.15.69:0/851042340 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22c80047e0 con 0x7f22d415e320 2024-04-25T10:59:33.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.195+0000 7f22dab8f640 1 -- 172.21.15.69:0/851042340 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22d4071be0 con 0x7f22d415e320 2024-04-25T10:59:33.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.195+0000 7f22d17fa640 1 -- 172.21.15.69:0/851042340 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c8004bd0 con 0x7f22d415e320 2024-04-25T10:59:33.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.195+0000 7f22dab8f640 1 -- 172.21.15.69:0/851042340 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22d40cb7d0 con 0x7f22d415e320 2024-04-25T10:59:33.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.196+0000 7f22dab8f640 1 -- 172.21.15.69:0/851042340 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22d4050740 con 0x7f22d415e320 2024-04-25T10:59:33.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.197+0000 7f22d17fa640 1 -- 172.21.15.69:0/851042340 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f22c8011070 con 0x7f22d415e320 2024-04-25T10:59:33.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.197+0000 7f22d17fa640 1 --2- 172.21.15.69:0/851042340 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f22c00744e0 0x7f22c00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:33.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.197+0000 7f22d892b640 1 --2- 172.21.15.69:0/851042340 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f22c00744e0 0x7f22c00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:33.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.197+0000 7f22d17fa640 1 -- 172.21.15.69:0/851042340 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f22c8094d10 con 0x7f22d415e320 2024-04-25T10:59:33.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.198+0000 7f22d892b640 1 --2- 172.21.15.69:0/851042340 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f22c00744e0 0x7f22c00769f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f22c4034000 tx=0x7f22c402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:33.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.202+0000 7f22d17fa640 1 -- 172.21.15.69:0/851042340 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f22c805f4b0 con 0x7f22d415e320 2024-04-25T10:59:33.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.415+0000 7f22dab8f640 1 -- 172.21.15.69:0/851042340 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f22d406ffb0 con 0x7f22d415e320 2024-04-25T10:59:33.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.416+0000 7f22d17fa640 1 -- 172.21.15.69:0/851042340 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7f22c805edb0 con 0x7f22d415e320 2024-04-25T10:59:33.417 INFO:teuthology.orchestra.run.smithi069.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:59:33.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.418+0000 7f22dab8f640 1 -- 172.21.15.69:0/851042340 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f22c00744e0 msgr2=0x7f22c00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:33.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.418+0000 7f22dab8f640 1 --2- 172.21.15.69:0/851042340 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f22c00744e0 0x7f22c00769f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f22c4034000 tx=0x7f22c402e000 comp rx=0 tx=0).stop 2024-04-25T10:59:33.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.418+0000 7f22dab8f640 1 -- 172.21.15.69:0/851042340 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22d415e320 msgr2=0x7f22d414fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:33.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.418+0000 7f22dab8f640 1 --2- 172.21.15.69:0/851042340 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22d415e320 0x7f22d414fe00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f22c800c460 tx=0x7f22c800c860 comp rx=0 tx=0).stop 2024-04-25T10:59:33.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.418+0000 7f22dab8f640 1 -- 172.21.15.69:0/851042340 shutdown_connections 2024-04-25T10:59:33.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.418+0000 7f22dab8f640 1 --2- 172.21.15.69:0/851042340 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f22c00744e0 0x7f22c00769f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:33.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.418+0000 7f22dab8f640 1 --2- 172.21.15.69:0/851042340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22d41f40d0 0x7f22d40c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:33.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.418+0000 7f22dab8f640 1 --2- 172.21.15.69:0/851042340 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22d415e320 0x7f22d414fe00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:33.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.418+0000 7f22dab8f640 1 --2- 172.21.15.69:0/851042340 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f22d40ce520 0x7f22d40c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:33.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.418+0000 7f22dab8f640 1 -- 172.21.15.69:0/851042340 >> 172.21.15.69:0/851042340 conn(0x7f22d409b7d0 msgr2=0x7f22d40d2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:33.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.418+0000 7f22dab8f640 1 -- 172.21.15.69:0/851042340 shutdown_connections 2024-04-25T10:59:33.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33.418+0000 7f22dab8f640 1 -- 172.21.15.69:0/851042340 wait complete. 2024-04-25T10:59:33.430 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:59:33.430 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:59:33.430 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi069.front.sepia.ceph.com 2024-04-25T10:59:33.430 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-25T10:59:33.430 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T10:59:33.430 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:59:33.430 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:59:33.430 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:59:33.430 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-25T10:59:33.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:33.430 DEBUG:teuthology.orchestra.run.smithi069:> ip addr 2024-04-25T10:59:33.445 INFO:teuthology.orchestra.run.smithi069.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:59:33.446 INFO:teuthology.orchestra.run.smithi069.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:59:33.446 INFO:teuthology.orchestra.run.smithi069.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:59:33.446 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:33.446 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 ::1/128 scope host 2024-04-25T10:59:33.446 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:33.446 INFO:teuthology.orchestra.run.smithi069.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:59:33.446 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:d8:d9:c2 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:59:33.446 INFO:teuthology.orchestra.run.smithi069.stdout: altname enp7s0f0 2024-04-25T10:59:33.446 INFO:teuthology.orchestra.run.smithi069.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:59:33.446 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:d8:d9:c3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:59:33.446 INFO:teuthology.orchestra.run.smithi069.stdout: altname enp7s0f1 2024-04-25T10:59:33.446 INFO:teuthology.orchestra.run.smithi069.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:59:33.446 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:8f:d7:36 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:59:33.447 INFO:teuthology.orchestra.run.smithi069.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:59:33.447 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:8f:d7:37 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:59:33.447 INFO:teuthology.orchestra.run.smithi069.stdout: inet 172.21.15.69/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T10:59:33.447 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft 42743sec preferred_lft 42743sec 2024-04-25T10:59:33.447 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 fe80::adbc:4e47:e7f9:8987/64 scope link noprefixroute 2024-04-25T10:59:33.447 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:33.447 INFO:teuthology.orchestra.run.smithi069.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:59:33.447 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 1a:fd:43:9b:af:c0 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:59:33.447 INFO:teuthology.orchestra.run.smithi069.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:59:33.447 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:33.447 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 fe80::10bd:ccff:fe05:fc3e/64 scope link 2024-04-25T10:59:33.447 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:33.447 INFO:teuthology.orchestra.run.smithi069.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:59:33.447 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 1a:fd:43:9b:af:c0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:59:33.447 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 fe80::18fd:43ff:fe9b:afc0/64 scope link tentative 2024-04-25T10:59:33.447 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:33.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:33.448 DEBUG:teuthology.orchestra.run.smithi069:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:59:33.507 INFO:teuthology.orchestra.run.smithi069.stdout:1 2024-04-25T10:59:33.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:33.509 DEBUG:teuthology.orchestra.run.smithi069:> route 2024-04-25T10:59:33.565 INFO:teuthology.orchestra.run.smithi069.stdout:Kernel IP routing table 2024-04-25T10:59:33.565 INFO:teuthology.orchestra.run.smithi069.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:59:33.565 INFO:teuthology.orchestra.run.smithi069.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:59:33.565 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:59:33.565 INFO:teuthology.orchestra.run.smithi069.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:59:33.565 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:59:33.566 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-04-25T10:59:33.566 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:59:33.566 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:59:33.566 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:59:33.566 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-04-25T10:59:33.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:59:33.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:33.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:33.688 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2024-04-25T10:59:33.703 INFO:teuthology.orchestra.run.smithi069.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:59:33.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:33.704 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list-id 2024-04-25T10:59:33.758 INFO:teuthology.orchestra.run.smithi069.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-25T10:59:33.758 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:59:33.758 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-04-25T10:59:33.758 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:59:33.759 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-25T10:59:33.759 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-04-25T10:59:33.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:59:33.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:33.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:33.875 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T10:59:33.898 INFO:teuthology.orchestra.run.smithi069.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:59:33.898 INFO:teuthology.orchestra.run.smithi069.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:59:33.898 INFO:teuthology.orchestra.run.smithi069.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:59:33.898 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:33.898 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 ::1/128 scope host 2024-04-25T10:59:33.898 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:33.899 INFO:teuthology.orchestra.run.smithi069.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:59:33.899 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 9a:04:e4:e1:6c:a9 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:59:33.899 INFO:teuthology.orchestra.run.smithi069.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:59:33.899 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:33.899 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 fe80::9804:e4ff:fee1:6ca9/64 scope link tentative 2024-04-25T10:59:33.899 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:33.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:33.900 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T10:59:33.964 INFO:teuthology.orchestra.run.smithi069.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:59:33.964 INFO:teuthology.orchestra.run.smithi069.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:59:33.964 INFO:teuthology.orchestra.run.smithi069.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:59:33.965 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:33.965 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 ::1/128 scope host 2024-04-25T10:59:33.965 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:33.965 INFO:teuthology.orchestra.run.smithi069.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:59:33.965 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 9a:04:e4:e1:6c:a9 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:59:33.965 INFO:teuthology.orchestra.run.smithi069.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:59:33.965 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:33.965 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 fe80::9804:e4ff:fee1:6ca9/64 scope link tentative 2024-04-25T10:59:33.965 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:33.966 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-25T10:59:33.967 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:59:33.967 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-04-25T10:59:33.967 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-25T10:59:33.967 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T10:59:33.967 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-25T10:59:33.967 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-25T10:59:33.967 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-25T10:59:33.967 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-04-25T10:59:34.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:59:34.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:34.114 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:59:34.114 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-04-25T10:59:34.114 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set brx.1 up 2024-04-25T10:59:34.114 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T10:59:34.114 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-04-25T10:59:34.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:59:34.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:34.189 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:59:34.189 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-25T10:59:34.203 INFO:teuthology.orchestra.run.smithi069.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-25T10:59:34.203 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:59:34.203 DEBUG:teuthology.orchestra.run.smithi069:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-25T10:59:34.257 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-04-25T10:59:34.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:34.258 DEBUG:teuthology.orchestra.run.smithi069:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:59:34.349 INFO:teuthology.orchestra.run.smithi069.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:59:34.350 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:59:34.358 INFO:teuthology.orchestra.run.smithi069.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:59:34.358 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:59:34.358 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: resolved to: "172.21.15.69:3300,172.21.15.146:3300,172.21.15.146:3301" 2024-04-25T10:59:34.358 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: trying mount with new device syntax: 2@ed0c8fe4-1bbb-4705-be72-eeea5bef859e.cephfs=/ 2024-04-25T10:59:34.359 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.69:3300/172.21.15.146:3300/172.21.15.146:3301" will pass to kernel 2024-04-25T10:59:34.359 INFO:teuthology.orchestra.run.smithi069.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-25T10:59:34.359 INFO:teuthology.orchestra.run.smithi069.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:59:34.359 INFO:teuthology.orchestra.run.smithi069.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:59:34.359 INFO:teuthology.orchestra.run.smithi069.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:59:34.359 INFO:teuthology.orchestra.run.smithi069.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:59:34.361 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:59:34.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:34.361 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-25T10:59:34.383 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:59:34.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T10:59:34.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:34.478 INFO:teuthology.orchestra.run.smithi069.stdout:client4579 2024-04-25T10:59:34.479 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:59:34.479 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:59:34.494 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:59:34.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T10:59:34.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:34.589 INFO:teuthology.orchestra.run.smithi069.stdout:client4579 2024-04-25T10:59:34.589 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:59:34.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T10:59:34.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:34.645 INFO:teuthology.orchestra.run.smithi069.stdout:client4579 2024-04-25T10:59:34.645 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/ed0c8fe4-1bbb-4705-be72-eeea5bef859e.client4579/status') 2024-04-25T10:59:34.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:59:34.701 INFO:teuthology.orchestra.run.smithi069.stdout:instance: client.4579 (3)172.21.15.69:0/3677389486 2024-04-25T10:59:34.702 INFO:teuthology.orchestra.run.smithi069.stdout:blocklisted: false 2024-04-25T10:59:34.702 INFO:teuthology.orchestra.run.smithi069.stderr:0+1 records in 2024-04-25T10:59:34.702 INFO:teuthology.orchestra.run.smithi069.stderr:0+1 records out 2024-04-25T10:59:34.702 INFO:teuthology.orchestra.run.smithi069.stderr:70 bytes copied, 2.0977e-05 s, 3.3 MB/s 2024-04-25T10:59:34.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:34.708 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-04-25T10:59:34.709 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:59:34.709 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T10:59:34.709 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:59:34.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.816+0000 7f5c871c5640 1 Processor -- start 2024-04-25T10:59:34.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.816+0000 7f5c871c5640 1 -- start start 2024-04-25T10:59:34.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.817+0000 7f5c871c5640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f5c800cf770 0x7f5c80162590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:34.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.817+0000 7f5c871c5640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5c80162ad0 0x7f5c800b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:34.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.817+0000 7f5c871c5640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c800cfc30 con 0x7f5c800b91d0 2024-04-25T10:59:34.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.817+0000 7f5c871c5640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f5c800d1ff0 con 0x7f5c800cf770 2024-04-25T10:59:34.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.817+0000 7f5c871c5640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f5c800d2890 con 0x7f5c80162ad0 2024-04-25T10:59:34.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.817+0000 7f5c84f61640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f5c800b91d0 0x7f5c80161e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:57284/0 (socket says 172.21.15.69:57284) 2024-04-25T10:59:34.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.817+0000 7f5c84f61640 1 -- 172.21.15.69:0/701771335 learned_addr learned my addr 172.21.15.69:0/701771335 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:34.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.817+0000 7f5c7ffff640 1 --2- 172.21.15.69:0/701771335 >> v2:172.21.15.69:3300/0 conn(0x7f5c800cf770 0x7f5c80162590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:34.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.818+0000 7f5c7f7fe640 1 -- 172.21.15.69:0/701771335 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1091645266 0 0) 0x7f5c800cfc30 con 0x7f5c800b91d0 2024-04-25T10:59:34.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.818+0000 7f5c85762640 1 --2- 172.21.15.69:0/701771335 >> v2:172.21.15.146:3300/0 conn(0x7f5c80162ad0 0x7f5c800b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:34.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.818+0000 7f5c7f7fe640 1 -- 172.21.15.69:0/701771335 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c70003520 con 0x7f5c800b91d0 2024-04-25T10:59:34.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.818+0000 7f5c7ffff640 1 -- 172.21.15.69:0/701771335 >> v2:172.21.15.146:3300/0 conn(0x7f5c80162ad0 msgr2=0x7f5c800b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:34.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.818+0000 7f5c7ffff640 1 --2- 172.21.15.69:0/701771335 >> v2:172.21.15.146:3300/0 conn(0x7f5c80162ad0 0x7f5c800b0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:34.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.818+0000 7f5c7ffff640 1 -- 172.21.15.69:0/701771335 >> v1:172.21.15.69:6789/0 conn(0x7f5c800b91d0 legacy=0x7f5c80161e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:34.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.818+0000 7f5c7ffff640 1 -- 172.21.15.69:0/701771335 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c800d2240 con 0x7f5c800cf770 2024-04-25T10:59:34.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.818+0000 7f5c7ffff640 1 --2- 172.21.15.69:0/701771335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c800cf770 0x7f5c80162590 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5c800caa70 tx=0x7f5c6c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ce23fdd803c180 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:34.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.818+0000 7f5c7f7fe640 1 -- 172.21.15.69:0/701771335 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c037070 con 0x7f5c800cf770 2024-04-25T10:59:34.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.819+0000 7f5c7f7fe640 1 -- 172.21.15.69:0/701771335 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c6c032440 con 0x7f5c800cf770 2024-04-25T10:59:34.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.819+0000 7f5c7f7fe640 1 -- 172.21.15.69:0/701771335 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c02ae60 con 0x7f5c800cf770 2024-04-25T10:59:34.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.819+0000 7f5c871c5640 1 -- 172.21.15.69:0/701771335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c800cf770 msgr2=0x7f5c80162590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:34.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.819+0000 7f5c871c5640 1 --2- 172.21.15.69:0/701771335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c800cf770 0x7f5c80162590 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5c800caa70 tx=0x7f5c6c02a110 comp rx=0 tx=0).stop 2024-04-25T10:59:34.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.819+0000 7f5c871c5640 1 -- 172.21.15.69:0/701771335 shutdown_connections 2024-04-25T10:59:34.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.819+0000 7f5c871c5640 1 --2- 172.21.15.69:0/701771335 >> v2:172.21.15.146:3300/0 conn(0x7f5c80162ad0 0x7f5c800b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:34.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.819+0000 7f5c871c5640 1 --2- 172.21.15.69:0/701771335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c800cf770 0x7f5c80162590 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:34.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.819+0000 7f5c871c5640 1 -- 172.21.15.69:0/701771335 >> 172.21.15.69:0/701771335 conn(0x7f5c800d3aa0 msgr2=0x7f5c800d3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:34.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.819+0000 7f5c871c5640 1 -- 172.21.15.69:0/701771335 shutdown_connections 2024-04-25T10:59:34.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.819+0000 7f5c871c5640 1 -- 172.21.15.69:0/701771335 wait complete. 2024-04-25T10:59:34.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.821+0000 7f5c871c5640 1 Processor -- start 2024-04-25T10:59:34.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.821+0000 7f5c871c5640 1 -- start start 2024-04-25T10:59:34.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.822+0000 7f5c871c5640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c800b91d0 0x7f5c8006eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:34.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.822+0000 7f5c871c5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c80162ad0 0x7f5c800ca080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:34.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.822+0000 7f5c871c5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c80160b00 0x7f5c80160f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:34.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.822+0000 7f5c871c5640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5c800b24a0 con 0x7f5c80162ad0 2024-04-25T10:59:34.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.822+0000 7f5c871c5640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5c800b2320 con 0x7f5c80160b00 2024-04-25T10:59:34.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.822+0000 7f5c871c5640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5c800b0bd0 con 0x7f5c800b91d0 2024-04-25T10:59:34.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.823+0000 7f5c7ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c80162ad0 0x7f5c800ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:34.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.823+0000 7f5c7ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c80162ad0 0x7f5c800ca080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56536/0 (socket says 172.21.15.69:56536) 2024-04-25T10:59:34.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.823+0000 7f5c7ffff640 1 -- 172.21.15.69:0/764147206 learned_addr learned my addr 172.21.15.69:0/764147206 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:34.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.823+0000 7f5c84f61640 1 --2- 172.21.15.69:0/764147206 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c800b91d0 0x7f5c8006eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:34.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.823+0000 7f5c85762640 1 --2- 172.21.15.69:0/764147206 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c80160b00 0x7f5c80160f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:34.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.823+0000 7f5c7ffff640 1 -- 172.21.15.69:0/764147206 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c800b91d0 msgr2=0x7f5c8006eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:34.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.823+0000 7f5c7ffff640 1 --2- 172.21.15.69:0/764147206 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c800b91d0 0x7f5c8006eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:34.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.823+0000 7f5c7ffff640 1 -- 172.21.15.69:0/764147206 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c80160b00 msgr2=0x7f5c80160f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:34.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.823+0000 7f5c7ffff640 1 --2- 172.21.15.69:0/764147206 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c80160b00 0x7f5c80160f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:34.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.823+0000 7f5c7ffff640 1 -- 172.21.15.69:0/764147206 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c80161450 con 0x7f5c80162ad0 2024-04-25T10:59:34.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.823+0000 7f5c7ffff640 1 --2- 172.21.15.69:0/764147206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c80162ad0 0x7f5c800ca080 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c000c00 tx=0x7f5c6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:34.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.823+0000 7f5c7d7fa640 1 -- 172.21.15.69:0/764147206 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c037040 con 0x7f5c80162ad0 2024-04-25T10:59:34.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.823+0000 7f5c7d7fa640 1 -- 172.21.15.69:0/764147206 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c6c042070 con 0x7f5c80162ad0 2024-04-25T10:59:34.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.823+0000 7f5c871c5640 1 -- 172.21.15.69:0/764147206 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c80160870 con 0x7f5c80162ad0 2024-04-25T10:59:34.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.823+0000 7f5c7d7fa640 1 -- 172.21.15.69:0/764147206 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c030960 con 0x7f5c80162ad0 2024-04-25T10:59:34.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.823+0000 7f5c871c5640 1 -- 172.21.15.69:0/764147206 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c800c9930 con 0x7f5c80162ad0 2024-04-25T10:59:34.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.824+0000 7f5c871c5640 1 -- 172.21.15.69:0/764147206 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c80050740 con 0x7f5c80162ad0 2024-04-25T10:59:34.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.825+0000 7f5c7d7fa640 1 -- 172.21.15.69:0/764147206 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5c6c030b00 con 0x7f5c80162ad0 2024-04-25T10:59:34.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.825+0000 7f5c7d7fa640 1 --2- 172.21.15.69:0/764147206 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5c640744b0 0x7f5c640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:34.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.825+0000 7f5c7d7fa640 1 -- 172.21.15.69:0/764147206 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5c6c0b5910 con 0x7f5c80162ad0 2024-04-25T10:59:34.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.825+0000 7f5c84f61640 1 --2- 172.21.15.69:0/764147206 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5c640744b0 0x7f5c640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:34.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.826+0000 7f5c84f61640 1 --2- 172.21.15.69:0/764147206 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5c640744b0 0x7f5c640769c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5c74005880 tx=0x7f5c74009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:34.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:34.830+0000 7f5c7d7fa640 1 -- 172.21.15.69:0/764147206 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5c6c04c020 con 0x7f5c80162ad0 2024-04-25T10:59:35.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:35.040+0000 7f5c871c5640 1 -- 172.21.15.69:0/764147206 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5c800d4700 con 0x7f5c80162ad0 2024-04-25T10:59:35.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:35.041+0000 7f5c7d7fa640 1 -- 172.21.15.69:0/764147206 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7f5c80161450 con 0x7f5c80162ad0 2024-04-25T10:59:35.042 INFO:teuthology.orchestra.run.smithi069.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:59:35.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:35.043+0000 7f5c871c5640 1 -- 172.21.15.69:0/764147206 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5c640744b0 msgr2=0x7f5c640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:35.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:35.043+0000 7f5c871c5640 1 --2- 172.21.15.69:0/764147206 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5c640744b0 0x7f5c640769c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5c74005880 tx=0x7f5c74009040 comp rx=0 tx=0).stop 2024-04-25T10:59:35.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:35.043+0000 7f5c871c5640 1 -- 172.21.15.69:0/764147206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c80162ad0 msgr2=0x7f5c800ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:35.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:35.043+0000 7f5c871c5640 1 --2- 172.21.15.69:0/764147206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c80162ad0 0x7f5c800ca080 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c000c00 tx=0x7f5c6c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:59:35.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:35.043+0000 7f5c871c5640 1 -- 172.21.15.69:0/764147206 shutdown_connections 2024-04-25T10:59:35.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:35.043+0000 7f5c871c5640 1 --2- 172.21.15.69:0/764147206 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5c640744b0 0x7f5c640769c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:35.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:35.043+0000 7f5c871c5640 1 --2- 172.21.15.69:0/764147206 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c80160b00 0x7f5c80160f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:35.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:35.043+0000 7f5c871c5640 1 --2- 172.21.15.69:0/764147206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c80162ad0 0x7f5c800ca080 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:35.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:35.043+0000 7f5c871c5640 1 --2- 172.21.15.69:0/764147206 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c800b91d0 0x7f5c8006eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:35.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:35.043+0000 7f5c871c5640 1 -- 172.21.15.69:0/764147206 >> 172.21.15.69:0/764147206 conn(0x7f5c800d3aa0 msgr2=0x7f5c80071ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:35.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:35.043+0000 7f5c871c5640 1 -- 172.21.15.69:0/764147206 shutdown_connections 2024-04-25T10:59:35.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:35.043+0000 7f5c871c5640 1 -- 172.21.15.69:0/764147206 wait complete. 2024-04-25T10:59:35.054 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:59:35.055 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:59:35.055 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi146.front.sepia.ceph.com 2024-04-25T10:59:35.055 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T10:59:35.055 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T10:59:35.055 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:59:35.055 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:59:35.055 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:59:35.055 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T10:59:35.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:35.055 DEBUG:teuthology.orchestra.run.smithi146:> ip addr 2024-04-25T10:59:35.074 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:59:35.074 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:59:35.074 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:59:35.074 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:35.074 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2024-04-25T10:59:35.074 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:35.074 INFO:teuthology.orchestra.run.smithi146.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:59:35.074 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6e brd ff:ff:ff:ff:ff:ff 2024-04-25T10:59:35.074 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f0 2024-04-25T10:59:35.074 INFO:teuthology.orchestra.run.smithi146.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:59:35.074 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:59:35.075 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f1 2024-04-25T10:59:35.075 INFO:teuthology.orchestra.run.smithi146.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:59:35.075 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:59:35.075 INFO:teuthology.orchestra.run.smithi146.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:59:35.075 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:59:35.075 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.21.15.146/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T10:59:35.075 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft 42763sec preferred_lft 42763sec 2024-04-25T10:59:35.075 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::baf5:b983:9020:4ba2/64 scope link noprefixroute 2024-04-25T10:59:35.075 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:35.075 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T10:59:35.076 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:59:35.076 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-04-25T10:59:35.076 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link add name ceph-brx type bridge 2024-04-25T10:59:35.076 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip addr flush dev ceph-brx 2024-04-25T10:59:35.076 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set ceph-brx up 2024-04-25T10:59:35.076 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T10:59:35.076 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-04-25T10:59:35.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:59:35.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:35.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:35.261 DEBUG:teuthology.orchestra.run.smithi146:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:59:35.285 INFO:teuthology.orchestra.run.smithi146.stdout:1 2024-04-25T10:59:35.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:35.286 DEBUG:teuthology.orchestra.run.smithi146:> route 2024-04-25T10:59:35.348 INFO:teuthology.orchestra.run.smithi146.stdout:Kernel IP routing table 2024-04-25T10:59:35.349 INFO:teuthology.orchestra.run.smithi146.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:59:35.349 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:59:35.349 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:59:35.349 INFO:teuthology.orchestra.run.smithi146.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:59:35.350 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:59:35.350 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-04-25T10:59:35.350 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:59:35.350 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:59:35.350 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:59:35.350 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-04-25T10:59:35.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:59:35.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:35.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:35.598 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-04-25T10:59:35.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:35.613 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list-id 2024-04-25T10:59:35.668 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:59:35.668 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-04-25T10:59:35.668 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:59:35.668 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T10:59:35.668 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-04-25T10:59:35.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:59:35.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:35.784 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T10:59:35.784 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:59:35.785 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-04-25T10:59:35.785 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T10:59:35.785 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T10:59:35.785 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T10:59:35.785 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T10:59:35.785 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T10:59:35.785 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-04-25T10:59:35.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:59:35.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:35.906 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:59:35.906 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-04-25T10:59:35.906 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set brx.0 up 2024-04-25T10:59:35.906 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T10:59:35.906 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-04-25T10:59:35.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:59:35.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:35.981 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:59:35.981 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T10:59:35.995 INFO:teuthology.orchestra.run.smithi146.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T10:59:35.995 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:59:35.996 DEBUG:teuthology.orchestra.run.smithi146:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T10:59:36.050 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-25T10:59:36.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:36.050 DEBUG:teuthology.orchestra.run.smithi146:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:59:36.148 INFO:teuthology.orchestra.run.smithi146.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:59:36.149 INFO:teuthology.orchestra.run.smithi146.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:59:36.253 INFO:teuthology.orchestra.run.smithi146.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:59:36.253 INFO:teuthology.orchestra.run.smithi146.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:59:36.253 INFO:teuthology.orchestra.run.smithi146.stdout:mount.ceph: resolved to: "172.21.15.69:3300,172.21.15.146:3300,172.21.15.146:3301" 2024-04-25T10:59:36.253 INFO:teuthology.orchestra.run.smithi146.stdout:mount.ceph: trying mount with new device syntax: 1@ed0c8fe4-1bbb-4705-be72-eeea5bef859e.cephfs=/ 2024-04-25T10:59:36.253 INFO:teuthology.orchestra.run.smithi146.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.69:3300/172.21.15.146:3300/172.21.15.146:3301" will pass to kernel 2024-04-25T10:59:36.254 INFO:teuthology.orchestra.run.smithi146.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-25T10:59:36.254 INFO:teuthology.orchestra.run.smithi146.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:59:36.254 INFO:teuthology.orchestra.run.smithi146.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:59:36.254 INFO:teuthology.orchestra.run.smithi146.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:59:36.254 INFO:teuthology.orchestra.run.smithi146.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:59:36.255 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:59:36.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:36.256 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T10:59:36.279 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:59:36.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T10:59:36.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:36.374 INFO:teuthology.orchestra.run.smithi146.stdout:client4451 2024-04-25T10:59:36.374 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T10:59:36.374 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:59:36.390 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:59:36.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T10:59:36.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:36.485 INFO:teuthology.orchestra.run.smithi146.stdout:client4451 2024-04-25T10:59:36.485 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:59:36.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T10:59:36.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:36.540 INFO:teuthology.orchestra.run.smithi146.stdout:client4451 2024-04-25T10:59:36.540 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/ed0c8fe4-1bbb-4705-be72-eeea5bef859e.client4451/status') 2024-04-25T10:59:36.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:59:36.597 INFO:teuthology.orchestra.run.smithi146.stdout:instance: client.4451 (3)192.168.0.1:0/1659182253 2024-04-25T10:59:36.597 INFO:teuthology.orchestra.run.smithi146.stdout:blocklisted: false 2024-04-25T10:59:36.598 INFO:teuthology.orchestra.run.smithi146.stderr:0+1 records in 2024-04-25T10:59:36.598 INFO:teuthology.orchestra.run.smithi146.stderr:0+1 records out 2024-04-25T10:59:36.598 INFO:teuthology.orchestra.run.smithi146.stderr:69 bytes copied, 3.8461e-05 s, 1.8 MB/s 2024-04-25T10:59:36.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:36.604 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-04-25T10:59:36.605 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:59:36.605 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T10:59:36.605 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:59:36.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.725+0000 7fdd6ad7a640 1 Processor -- start 2024-04-25T10:59:36.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.725+0000 7fdd6ad7a640 1 -- start start 2024-04-25T10:59:36.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.726+0000 7fdd6ad7a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd640d4130 0x7fdd640d1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:36.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.726+0000 7fdd6ad7a640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fdd6406dcf0 0x7fdd640d22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:36.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.726+0000 7fdd6ad7a640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd640b3c10 con 0x7fdd6406d8e0 2024-04-25T10:59:36.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.727+0000 7fdd6ad7a640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fdd640b9520 con 0x7fdd6406dcf0 2024-04-25T10:59:36.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.727+0000 7fdd6ad7a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fdd640b4820 con 0x7fdd640d4130 2024-04-25T10:59:36.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.727+0000 7fdd69317640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fdd6406dcf0 0x7fdd640d22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:36.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.727+0000 7fdd69317640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fdd6406dcf0 0x7fdd640d22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43544/0 (socket says 172.21.15.69:43544) 2024-04-25T10:59:36.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.727+0000 7fdd69317640 1 -- 172.21.15.69:0/131271323 learned_addr learned my addr 172.21.15.69:0/131271323 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:36.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.727+0000 7fdd63fff640 1 --2- 172.21.15.69:0/131271323 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd640d4130 0x7fdd640d1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:36.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.727+0000 7fdd69317640 1 -- 172.21.15.69:0/131271323 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd640d4130 msgr2=0x7fdd640d1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:36.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.727+0000 7fdd69317640 1 --2- 172.21.15.69:0/131271323 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd640d4130 0x7fdd640d1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:36.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.727+0000 7fdd69317640 1 -- 172.21.15.69:0/131271323 >> v1:172.21.15.146:6789/0 conn(0x7fdd6406d8e0 legacy=0x7fdd640d3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:36.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.727+0000 7fdd69317640 1 -- 172.21.15.69:0/131271323 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd640b3ef0 con 0x7fdd6406dcf0 2024-04-25T10:59:36.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.728+0000 7fdd69317640 1 --2- 172.21.15.69:0/131271323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd6406dcf0 0x7fdd640d22a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c009e50 tx=0x7fdd5c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3b8620eada01e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:36.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.728+0000 7fdd637fe640 1 -- 172.21.15.69:0/131271323 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c036070 con 0x7fdd6406dcf0 2024-04-25T10:59:36.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.728+0000 7fdd637fe640 1 -- 172.21.15.69:0/131271323 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd5c031600 con 0x7fdd6406dcf0 2024-04-25T10:59:36.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.728+0000 7fdd637fe640 1 -- 172.21.15.69:0/131271323 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c0319d0 con 0x7fdd6406dcf0 2024-04-25T10:59:36.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.729+0000 7fdd6ad7a640 1 -- 172.21.15.69:0/131271323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd6406dcf0 msgr2=0x7fdd640d22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:36.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.729+0000 7fdd6ad7a640 1 --2- 172.21.15.69:0/131271323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd6406dcf0 0x7fdd640d22a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c009e50 tx=0x7fdd5c02a680 comp rx=0 tx=0).stop 2024-04-25T10:59:36.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.729+0000 7fdd6ad7a640 1 -- 172.21.15.69:0/131271323 shutdown_connections 2024-04-25T10:59:36.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.729+0000 7fdd6ad7a640 1 --2- 172.21.15.69:0/131271323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd6406dcf0 0x7fdd640d22a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:36.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.729+0000 7fdd6ad7a640 1 --2- 172.21.15.69:0/131271323 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd640d4130 0x7fdd640d1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:36.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.729+0000 7fdd6ad7a640 1 -- 172.21.15.69:0/131271323 >> 172.21.15.69:0/131271323 conn(0x7fdd6408d110 msgr2=0x7fdd6408d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:36.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.729+0000 7fdd6ad7a640 1 -- 172.21.15.69:0/131271323 shutdown_connections 2024-04-25T10:59:36.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.729+0000 7fdd6ad7a640 1 -- 172.21.15.69:0/131271323 wait complete. 2024-04-25T10:59:36.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.730+0000 7fdd6ad7a640 1 Processor -- start 2024-04-25T10:59:36.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.731+0000 7fdd6ad7a640 1 -- start start 2024-04-25T10:59:36.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.732+0000 7fdd6ad7a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd6406d8e0 0x7fdd641e6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:36.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.732+0000 7fdd6ad7a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd6406dcf0 0x7fdd6414df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:36.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.732+0000 7fdd6ad7a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd6414ec80 0x7fdd6414e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:36.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.732+0000 7fdd6ad7a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdd640b38e0 con 0x7fdd6406d8e0 2024-04-25T10:59:36.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.732+0000 7fdd6ad7a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdd640b44b0 con 0x7fdd6406dcf0 2024-04-25T10:59:36.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.732+0000 7fdd6ad7a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fdd640d27e0 con 0x7fdd6414ec80 2024-04-25T10:59:36.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.732+0000 7fdd68b16640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd6406d8e0 0x7fdd641e6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:36.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.732+0000 7fdd68b16640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd6406d8e0 0x7fdd641e6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43552/0 (socket says 172.21.15.69:43552) 2024-04-25T10:59:36.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.732+0000 7fdd68b16640 1 -- 172.21.15.69:0/1668060809 learned_addr learned my addr 172.21.15.69:0/1668060809 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:36.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.732+0000 7fdd63fff640 1 --2- 172.21.15.69:0/1668060809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd6406dcf0 0x7fdd6414df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:36.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.732+0000 7fdd69317640 1 --2- 172.21.15.69:0/1668060809 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd6414ec80 0x7fdd6414e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:36.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.732+0000 7fdd68b16640 1 -- 172.21.15.69:0/1668060809 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd6414ec80 msgr2=0x7fdd6414e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:36.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.732+0000 7fdd68b16640 1 --2- 172.21.15.69:0/1668060809 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd6414ec80 0x7fdd6414e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:36.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.732+0000 7fdd68b16640 1 -- 172.21.15.69:0/1668060809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd6406dcf0 msgr2=0x7fdd6414df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:36.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.732+0000 7fdd68b16640 1 --2- 172.21.15.69:0/1668060809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd6406dcf0 0x7fdd6414df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:36.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.732+0000 7fdd68b16640 1 -- 172.21.15.69:0/1668060809 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd640b7910 con 0x7fdd6406d8e0 2024-04-25T10:59:36.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.733+0000 7fdd68b16640 1 --2- 172.21.15.69:0/1668060809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd6406d8e0 0x7fdd641e6a30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400c440 tx=0x7fdd5400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:36.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.733+0000 7fdd617fa640 1 -- 172.21.15.69:0/1668060809 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd540052d0 con 0x7fdd6406d8e0 2024-04-25T10:59:36.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.733+0000 7fdd617fa640 1 -- 172.21.15.69:0/1668060809 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd54005470 con 0x7fdd6406d8e0 2024-04-25T10:59:36.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.733+0000 7fdd6ad7a640 1 -- 172.21.15.69:0/1668060809 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd640b7bf0 con 0x7fdd6406d8e0 2024-04-25T10:59:36.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.733+0000 7fdd617fa640 1 -- 172.21.15.69:0/1668060809 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd54007260 con 0x7fdd6406d8e0 2024-04-25T10:59:36.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.733+0000 7fdd6ad7a640 1 -- 172.21.15.69:0/1668060809 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd640b81b0 con 0x7fdd6406d8e0 2024-04-25T10:59:36.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.734+0000 7fdd6ad7a640 1 -- 172.21.15.69:0/1668060809 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd64050740 con 0x7fdd6406d8e0 2024-04-25T10:59:36.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.734+0000 7fdd617fa640 1 -- 172.21.15.69:0/1668060809 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdd54007400 con 0x7fdd6406d8e0 2024-04-25T10:59:36.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.735+0000 7fdd617fa640 1 --2- 172.21.15.69:0/1668060809 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fdd500744e0 0x7fdd500769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:36.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.735+0000 7fdd63fff640 1 --2- 172.21.15.69:0/1668060809 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fdd500744e0 0x7fdd500769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:36.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.735+0000 7fdd617fa640 1 -- 172.21.15.69:0/1668060809 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdd54094550 con 0x7fdd6406d8e0 2024-04-25T10:59:36.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.735+0000 7fdd63fff640 1 --2- 172.21.15.69:0/1668060809 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fdd500744e0 0x7fdd500769f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdd64152610 tx=0x7fdd58009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:36.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.739+0000 7fdd617fa640 1 -- 172.21.15.69:0/1668060809 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdd5405e720 con 0x7fdd6406d8e0 2024-04-25T10:59:36.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.953+0000 7fdd6ad7a640 1 -- 172.21.15.69:0/1668060809 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdd640714f0 con 0x7fdd6406d8e0 2024-04-25T10:59:36.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.953+0000 7fdd617fa640 1 -- 172.21.15.69:0/1668060809 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7fdd54062a20 con 0x7fdd6406d8e0 2024-04-25T10:59:36.954 INFO:teuthology.orchestra.run.smithi069.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:59:36.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.955+0000 7fdd6ad7a640 1 -- 172.21.15.69:0/1668060809 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fdd500744e0 msgr2=0x7fdd500769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:36.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.955+0000 7fdd6ad7a640 1 --2- 172.21.15.69:0/1668060809 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fdd500744e0 0x7fdd500769f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdd64152610 tx=0x7fdd58009190 comp rx=0 tx=0).stop 2024-04-25T10:59:36.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.955+0000 7fdd6ad7a640 1 -- 172.21.15.69:0/1668060809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd6406d8e0 msgr2=0x7fdd641e6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:36.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.955+0000 7fdd6ad7a640 1 --2- 172.21.15.69:0/1668060809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd6406d8e0 0x7fdd641e6a30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400c440 tx=0x7fdd5400c900 comp rx=0 tx=0).stop 2024-04-25T10:59:36.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.955+0000 7fdd6ad7a640 1 -- 172.21.15.69:0/1668060809 shutdown_connections 2024-04-25T10:59:36.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.955+0000 7fdd6ad7a640 1 --2- 172.21.15.69:0/1668060809 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fdd500744e0 0x7fdd500769f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:36.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.955+0000 7fdd6ad7a640 1 --2- 172.21.15.69:0/1668060809 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd6414ec80 0x7fdd6414e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:36.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.955+0000 7fdd6ad7a640 1 --2- 172.21.15.69:0/1668060809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd6406dcf0 0x7fdd6414df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:36.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.955+0000 7fdd6ad7a640 1 --2- 172.21.15.69:0/1668060809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd6406d8e0 0x7fdd641e6a30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:36.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.955+0000 7fdd6ad7a640 1 -- 172.21.15.69:0/1668060809 >> 172.21.15.69:0/1668060809 conn(0x7fdd6408d110 msgr2=0x7fdd640b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:36.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.955+0000 7fdd6ad7a640 1 -- 172.21.15.69:0/1668060809 shutdown_connections 2024-04-25T10:59:36.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:36.955+0000 7fdd6ad7a640 1 -- 172.21.15.69:0/1668060809 wait complete. 2024-04-25T10:59:36.968 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:59:36.968 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:59:36.968 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi146.front.sepia.ceph.com 2024-04-25T10:59:36.968 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-25T10:59:36.968 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T10:59:36.968 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:59:36.969 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:59:36.969 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:59:36.969 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-25T10:59:36.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:36.969 DEBUG:teuthology.orchestra.run.smithi146:> ip addr 2024-04-25T10:59:36.986 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:59:36.986 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:59:36.986 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:59:36.986 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:36.987 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2024-04-25T10:59:36.987 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:36.987 INFO:teuthology.orchestra.run.smithi146.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:59:36.987 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6e brd ff:ff:ff:ff:ff:ff 2024-04-25T10:59:36.987 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f0 2024-04-25T10:59:36.987 INFO:teuthology.orchestra.run.smithi146.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:59:36.988 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:59:36.988 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f1 2024-04-25T10:59:36.988 INFO:teuthology.orchestra.run.smithi146.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:59:36.988 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:59:36.988 INFO:teuthology.orchestra.run.smithi146.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:59:36.988 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:59:36.988 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.21.15.146/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T10:59:36.989 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft 42761sec preferred_lft 42761sec 2024-04-25T10:59:36.989 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::baf5:b983:9020:4ba2/64 scope link noprefixroute 2024-04-25T10:59:36.989 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:36.989 INFO:teuthology.orchestra.run.smithi146.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:59:36.989 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 16:9b:a0:44:7c:56 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:59:36.989 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:59:36.990 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:36.990 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::accf:52ff:fedf:61e1/64 scope link 2024-04-25T10:59:36.990 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:36.990 INFO:teuthology.orchestra.run.smithi146.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:59:36.990 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 16:9b:a0:44:7c:56 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:59:36.990 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::149b:a0ff:fe44:7c56/64 scope link tentative 2024-04-25T10:59:36.990 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:36.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:36.991 DEBUG:teuthology.orchestra.run.smithi146:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:59:37.049 INFO:teuthology.orchestra.run.smithi146.stdout:1 2024-04-25T10:59:37.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:37.051 DEBUG:teuthology.orchestra.run.smithi146:> route 2024-04-25T10:59:37.107 INFO:teuthology.orchestra.run.smithi146.stdout:Kernel IP routing table 2024-04-25T10:59:37.107 INFO:teuthology.orchestra.run.smithi146.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:59:37.107 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:59:37.107 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:59:37.107 INFO:teuthology.orchestra.run.smithi146.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:59:37.107 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:59:37.107 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-04-25T10:59:37.107 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:59:37.108 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:59:37.108 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:59:37.108 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-04-25T10:59:37.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:59:37.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:37.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:37.231 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-04-25T10:59:37.245 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T10:59:37.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:37.246 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list-id 2024-04-25T10:59:37.299 INFO:teuthology.orchestra.run.smithi146.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-25T10:59:37.300 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:59:37.300 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-04-25T10:59:37.300 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:59:37.300 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-25T10:59:37.300 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-04-25T10:59:37.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:59:37.408 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:37.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:37.414 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T10:59:37.437 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:59:37.437 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:59:37.437 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:59:37.437 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:37.437 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2024-04-25T10:59:37.437 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:37.437 INFO:teuthology.orchestra.run.smithi146.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:59:37.437 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether f6:a7:b3:64:b1:d8 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:59:37.438 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:59:37.438 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:37.438 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::f4a7:b3ff:fe64:b1d8/64 scope link 2024-04-25T10:59:37.438 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:37.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:37.439 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T10:59:37.502 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:59:37.502 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:59:37.502 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:59:37.503 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:37.503 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2024-04-25T10:59:37.503 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:37.503 INFO:teuthology.orchestra.run.smithi146.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:59:37.503 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether f6:a7:b3:64:b1:d8 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:59:37.503 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:59:37.503 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:37.503 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::f4a7:b3ff:fe64:b1d8/64 scope link 2024-04-25T10:59:37.503 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:59:37.504 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-25T10:59:37.504 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:59:37.504 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-04-25T10:59:37.504 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-25T10:59:37.505 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T10:59:37.505 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-25T10:59:37.505 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-25T10:59:37.505 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-25T10:59:37.505 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-04-25T10:59:37.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:59:37.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:37.651 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:59:37.651 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-04-25T10:59:37.651 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set brx.1 up 2024-04-25T10:59:37.651 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T10:59:37.651 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-04-25T10:59:37.699 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:59:37.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:37.726 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:59:37.726 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-25T10:59:37.740 INFO:teuthology.orchestra.run.smithi146.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-25T10:59:37.741 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:59:37.741 DEBUG:teuthology.orchestra.run.smithi146:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-25T10:59:37.795 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-04-25T10:59:37.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:37.796 DEBUG:teuthology.orchestra.run.smithi146:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:59:37.887 INFO:teuthology.orchestra.run.smithi146.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:59:37.887 INFO:teuthology.orchestra.run.smithi146.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:59:37.899 INFO:teuthology.orchestra.run.smithi146.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:59:37.899 INFO:teuthology.orchestra.run.smithi146.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:59:37.899 INFO:teuthology.orchestra.run.smithi146.stdout:mount.ceph: resolved to: "172.21.15.69:3300,172.21.15.146:3300,172.21.15.146:3301" 2024-04-25T10:59:37.899 INFO:teuthology.orchestra.run.smithi146.stdout:mount.ceph: trying mount with new device syntax: 3@ed0c8fe4-1bbb-4705-be72-eeea5bef859e.cephfs=/ 2024-04-25T10:59:37.899 INFO:teuthology.orchestra.run.smithi146.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.69:3300/172.21.15.146:3300/172.21.15.146:3301" will pass to kernel 2024-04-25T10:59:37.899 INFO:teuthology.orchestra.run.smithi146.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-25T10:59:37.899 INFO:teuthology.orchestra.run.smithi146.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:59:37.899 INFO:teuthology.orchestra.run.smithi146.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:59:37.899 INFO:teuthology.orchestra.run.smithi146.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:59:37.900 INFO:teuthology.orchestra.run.smithi146.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:59:37.901 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:59:37.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:37.901 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-25T10:59:37.924 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:59:38.014 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T10:59:38.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:38.021 INFO:teuthology.orchestra.run.smithi146.stdout:client4460 2024-04-25T10:59:38.021 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T10:59:38.021 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:59:38.038 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:59:38.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T10:59:38.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:38.134 INFO:teuthology.orchestra.run.smithi146.stdout:client4460 2024-04-25T10:59:38.135 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:59:38.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T10:59:38.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:38.190 INFO:teuthology.orchestra.run.smithi146.stdout:client4460 2024-04-25T10:59:38.190 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/ed0c8fe4-1bbb-4705-be72-eeea5bef859e.client4460/status') 2024-04-25T10:59:38.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:59:38.248 INFO:teuthology.orchestra.run.smithi146.stdout:instance: client.4460 (3)192.168.0.2:0/3460206861 2024-04-25T10:59:38.248 INFO:teuthology.orchestra.run.smithi146.stdout:blocklisted: false 2024-04-25T10:59:38.248 INFO:teuthology.orchestra.run.smithi146.stderr:0+1 records in 2024-04-25T10:59:38.248 INFO:teuthology.orchestra.run.smithi146.stderr:0+1 records out 2024-04-25T10:59:38.248 INFO:teuthology.orchestra.run.smithi146.stderr:69 bytes copied, 2.2144e-05 s, 3.1 MB/s 2024-04-25T10:59:38.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T10:59:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:59:38.255 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-25T10:59:38.276 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:59:38.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.397+0000 7f1561634640 1 Processor -- start 2024-04-25T10:59:38.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.397+0000 7f1561634640 1 -- start start 2024-04-25T10:59:38.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.398+0000 7f1561634640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f155c0b91d0 0x7f155c161e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:38.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.398+0000 7f1561634640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f155c0cf770 0x7f155c0b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:38.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.398+0000 7f1561634640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f155c0cfc30 con 0x7f155c1623c0 2024-04-25T10:59:38.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.398+0000 7f1561634640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f155c0d1ff0 con 0x7f155c0cf770 2024-04-25T10:59:38.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.398+0000 7f1561634640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f155c0d2890 con 0x7f155c0b91d0 2024-04-25T10:59:38.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.398+0000 7f1553fff640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f155c1623c0 0x7f155c1627d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:44256/0 (socket says 172.21.15.69:44256) 2024-04-25T10:59:38.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.398+0000 7f1553fff640 1 -- 172.21.15.69:0/2763250403 learned_addr learned my addr 172.21.15.69:0/2763250403 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:38.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.399+0000 7f155affd640 1 --2- 172.21.15.69:0/2763250403 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f155c0b91d0 0x7f155c161e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:38.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.399+0000 7f155b7fe640 1 --2- 172.21.15.69:0/2763250403 >> v2:172.21.15.146:3300/0 conn(0x7f155c0cf770 0x7f155c0b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:38.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.399+0000 7f155a7fc640 1 -- 172.21.15.69:0/2763250403 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3376774180 0 0) 0x7f155c0cfc30 con 0x7f155c1623c0 2024-04-25T10:59:38.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.399+0000 7f155a7fc640 1 -- 172.21.15.69:0/2763250403 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1548003520 con 0x7f155c1623c0 2024-04-25T10:59:38.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.399+0000 7f155a7fc640 1 -- 172.21.15.69:0/2763250403 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4009056414 0 0) 0x7f1548003520 con 0x7f155c1623c0 2024-04-25T10:59:38.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.399+0000 7f155b7fe640 1 -- 172.21.15.69:0/2763250403 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f155c0b91d0 msgr2=0x7f155c161e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:38.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.399+0000 7f155b7fe640 1 --2- 172.21.15.69:0/2763250403 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f155c0b91d0 0x7f155c161e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:38.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.399+0000 7f155b7fe640 1 -- 172.21.15.69:0/2763250403 >> v1:172.21.15.69:6789/0 conn(0x7f155c1623c0 legacy=0x7f155c1627d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:38.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.399+0000 7f155b7fe640 1 -- 172.21.15.69:0/2763250403 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f155c0d2240 con 0x7f155c0cf770 2024-04-25T10:59:38.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.399+0000 7f155affd640 1 --2- 172.21.15.69:0/2763250403 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f155c0b91d0 0x7f155c161e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:59:38.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.400+0000 7f155b7fe640 1 --2- 172.21.15.69:0/2763250403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f155c0cf770 0x7f155c0b0690 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1554009d90 tx=0x7f155402a560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38b96a604708dee1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:38.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.400+0000 7f155a7fc640 1 -- 172.21.15.69:0/2763250403 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1544001670 con 0x7f155c0cf770 2024-04-25T10:59:38.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.400+0000 7f155a7fc640 1 -- 172.21.15.69:0/2763250403 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1554007540 con 0x7f155c0cf770 2024-04-25T10:59:38.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.400+0000 7f155a7fc640 1 -- 172.21.15.69:0/2763250403 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1554007910 con 0x7f155c0cf770 2024-04-25T10:59:38.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.401+0000 7f1561634640 1 -- 172.21.15.69:0/2763250403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f155c0cf770 msgr2=0x7f155c0b0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:38.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.401+0000 7f1561634640 1 --2- 172.21.15.69:0/2763250403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f155c0cf770 0x7f155c0b0690 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1554009d90 tx=0x7f155402a560 comp rx=0 tx=0).stop 2024-04-25T10:59:38.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.401+0000 7f1561634640 1 -- 172.21.15.69:0/2763250403 shutdown_connections 2024-04-25T10:59:38.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.401+0000 7f1561634640 1 --2- 172.21.15.69:0/2763250403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f155c0cf770 0x7f155c0b0690 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:38.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.401+0000 7f1561634640 1 --2- 172.21.15.69:0/2763250403 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f155c0b91d0 0x7f155c161e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:38.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.401+0000 7f1561634640 1 -- 172.21.15.69:0/2763250403 >> 172.21.15.69:0/2763250403 conn(0x7f155c0d3aa0 msgr2=0x7f155c0d3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:38.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.401+0000 7f1561634640 1 -- 172.21.15.69:0/2763250403 shutdown_connections 2024-04-25T10:59:38.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.401+0000 7f1561634640 1 -- 172.21.15.69:0/2763250403 wait complete. 2024-04-25T10:59:38.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.403+0000 7f1561634640 1 Processor -- start 2024-04-25T10:59:38.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.403+0000 7f1561634640 1 -- start start 2024-04-25T10:59:38.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.404+0000 7f1561634640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f155c0b91d0 0x7f155c0cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:38.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.404+0000 7f1561634640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f155c1623c0 0x7f155c159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:38.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.404+0000 7f1561634640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f155c15a370 0x7f155c0c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:38.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.404+0000 7f1561634640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f155c0b24a0 con 0x7f155c0b91d0 2024-04-25T10:59:38.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.404+0000 7f1561634640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f155c0b2320 con 0x7f155c1623c0 2024-04-25T10:59:38.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.404+0000 7f1561634640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f155c0b0bd0 con 0x7f155c15a370 2024-04-25T10:59:38.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.404+0000 7f155affd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f155c0b91d0 0x7f155c0cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:38.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.404+0000 7f155affd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f155c0b91d0 0x7f155c0cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43556/0 (socket says 172.21.15.69:43556) 2024-04-25T10:59:38.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.404+0000 7f155affd640 1 -- 172.21.15.69:0/1616436427 learned_addr learned my addr 172.21.15.69:0/1616436427 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:38.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.404+0000 7f1553fff640 1 --2- 172.21.15.69:0/1616436427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f155c1623c0 0x7f155c159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:38.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.404+0000 7f155b7fe640 1 --2- 172.21.15.69:0/1616436427 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f155c15a370 0x7f155c0c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:38.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.404+0000 7f155affd640 1 -- 172.21.15.69:0/1616436427 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f155c15a370 msgr2=0x7f155c0c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:38.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.404+0000 7f155affd640 1 --2- 172.21.15.69:0/1616436427 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f155c15a370 0x7f155c0c8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:38.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.404+0000 7f155affd640 1 -- 172.21.15.69:0/1616436427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f155c1623c0 msgr2=0x7f155c159ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:38.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.404+0000 7f155affd640 1 --2- 172.21.15.69:0/1616436427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f155c1623c0 0x7f155c159ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:38.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.404+0000 7f155affd640 1 -- 172.21.15.69:0/1616436427 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f155c0cd8f0 con 0x7f155c0b91d0 2024-04-25T10:59:38.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.405+0000 7f155affd640 1 --2- 172.21.15.69:0/1616436427 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f155c0b91d0 0x7f155c0cd3b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f154c00e960 tx=0x7f154c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:38.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.405+0000 7f15537fe640 1 -- 172.21.15.69:0/1616436427 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c017070 con 0x7f155c0b91d0 2024-04-25T10:59:38.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.405+0000 7f15537fe640 1 -- 172.21.15.69:0/1616436427 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f154c00ca00 con 0x7f155c0b91d0 2024-04-25T10:59:38.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.405+0000 7f15537fe640 1 -- 172.21.15.69:0/1616436427 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c00cdd0 con 0x7f155c0b91d0 2024-04-25T10:59:38.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.405+0000 7f1561634640 1 -- 172.21.15.69:0/1616436427 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f155c15faf0 con 0x7f155c0b91d0 2024-04-25T10:59:38.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.405+0000 7f1561634640 1 -- 172.21.15.69:0/1616436427 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f155c0ccbb0 con 0x7f155c0b91d0 2024-04-25T10:59:38.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.406+0000 7f1561634640 1 -- 172.21.15.69:0/1616436427 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f155c050740 con 0x7f155c0b91d0 2024-04-25T10:59:38.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.406+0000 7f15537fe640 1 -- 172.21.15.69:0/1616436427 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f154c01a070 con 0x7f155c0b91d0 2024-04-25T10:59:38.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.407+0000 7f15537fe640 1 --2- 172.21.15.69:0/1616436427 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f153c0744b0 0x7f153c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:38.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.407+0000 7f15537fe640 1 -- 172.21.15.69:0/1616436427 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f154c094ff0 con 0x7f155c0b91d0 2024-04-25T10:59:38.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.407+0000 7f1553fff640 1 --2- 172.21.15.69:0/1616436427 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f153c0744b0 0x7f153c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:38.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.407+0000 7f1553fff640 1 --2- 172.21.15.69:0/1616436427 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f153c0744b0 0x7f153c0769c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f155c0b6060 tx=0x7f1544000fe0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:38.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.412+0000 7f15537fe640 1 -- 172.21.15.69:0/1616436427 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f154c05f1c0 con 0x7f155c0b91d0 2024-04-25T10:59:38.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.591+0000 7f1561634640 1 -- 172.21.15.69:0/1616436427 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f155c0d4700 con 0x7f155c0b91d0 2024-04-25T10:59:38.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.592+0000 7f15537fe640 1 -- 172.21.15.69:0/1616436427 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f154c0634c0 con 0x7f155c0b91d0 2024-04-25T10:59:38.594 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:38.594 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":20,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T10:59:20.630151+0000","last_up_change":"2024-04-25T10:59:05.546839+0000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:59:06.422674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:59:16.727980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:59:17.938486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6825","nonce":1876101065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6827","nonce":1876101065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6831","nonce":1876101065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6829","nonce":1876101065}]},"public_addr":"172.21.15.69:6825/1876101065","cluster_addr":"172.21.15.69:6827/1876101065","heartbeat_back_addr":"172.21.15.69:6831/1876101065","heartbeat_front_addr":"172.21.15.69:6829/1876101065","state":["exists","up"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6801","nonce":1806438563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6803","nonce":1806438563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6807","nonce":1806438563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6805","nonce":1806438563}]},"public_addr":"172.21.15.69:6801/1806438563","cluster_addr":"172.21.15.69:6803/1806438563","heartbeat_back_addr":"172.21.15.69:6807/1806438563","heartbeat_front_addr":"172.21.15.69:6805/1806438563","state":["exists","up"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6817","nonce":1593617983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6819","nonce":1593617983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6823","nonce":1593617983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6821","nonce":1593617983}]},"public_addr":"172.21.15.69:6817/1593617983","cluster_addr":"172.21.15.69:6819/1593617983","heartbeat_back_addr":"172.21.15.69:6823/1593617983","heartbeat_front_addr":"172.21.15.69:6821/1593617983","state":["exists","up"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6809","nonce":3950661294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6811","nonce":3950661294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6815","nonce":3950661294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6813","nonce":3950661294}]},"public_addr":"172.21.15.69:6809/3950661294","cluster_addr":"172.21.15.69:6811/3950661294","heartbeat_back_addr":"172.21.15.69:6815/3950661294","heartbeat_front_addr":"172.21.15.69:6813/3950661294","state":["exists","up"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6801","nonce":2508929872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6803","nonce":2508929872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6807","nonce":2508929872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6805","nonce":2508929872}]},"public_addr":"172.21.15.146:6801/2508929872","cluster_addr":"172.21.15.146:6803/2508929872","heartbeat_back_addr":"172.21.15.146:6807/2508929872","heartbeat_front_addr":"172.21.15.146:6805/2508929872","state":["exists","up"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6811","nonce":1310911138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6816","nonce":1310911138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6823","nonce":1310911138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6821","nonce":1310911138}]},"public_addr":"172.21.15.146:6811/1310911138","cluster_addr":"172.21.15.146:6816/1310911138","heartbeat_back_addr":"172.21.15.146:6823/1310911138","heartbeat_front_addr":"172.21.15.146:6821/1310911138","state":["exists","up"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6810","nonce":634688021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6813","nonce":634688021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6820","nonce":634688021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6817","nonce":634688021}]},"public_addr":"172.21.15.146:6810/634688021","cluster_addr":"172.21.15.146:6813/634688021","heartbeat_back_addr":"172.21.15.146:6820/634688021","heartbeat_front_addr":"172.21.15.146:6817/634688021","state":["exists","up"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6825","nonce":3814215935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6827","nonce":3814215935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6831","nonce":3814215935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6829","nonce":3814215935}]},"public_addr":"172.21.15.146:6825/3814215935","cluster_addr":"172.21.15.146:6827/3814215935","heartbeat_back_addr":"172.21.15.146:6831/3814215935","heartbeat_front_addr":"172.21.15.146:6829/3814215935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.820200+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.850189+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.764560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.866248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.807897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.869881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.782310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.827131+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:59:38.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.595+0000 7f1561634640 1 -- 172.21.15.69:0/1616436427 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f153c0744b0 msgr2=0x7f153c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:38.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.595+0000 7f1561634640 1 --2- 172.21.15.69:0/1616436427 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f153c0744b0 0x7f153c0769c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f155c0b6060 tx=0x7f1544000fe0 comp rx=0 tx=0).stop 2024-04-25T10:59:38.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.595+0000 7f1561634640 1 -- 172.21.15.69:0/1616436427 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f155c0b91d0 msgr2=0x7f155c0cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:38.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.595+0000 7f1561634640 1 --2- 172.21.15.69:0/1616436427 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f155c0b91d0 0x7f155c0cd3b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f154c00e960 tx=0x7f154c00ee20 comp rx=0 tx=0).stop 2024-04-25T10:59:38.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.595+0000 7f1561634640 1 -- 172.21.15.69:0/1616436427 shutdown_connections 2024-04-25T10:59:38.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.595+0000 7f1561634640 1 --2- 172.21.15.69:0/1616436427 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f153c0744b0 0x7f153c0769c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:38.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.595+0000 7f1561634640 1 --2- 172.21.15.69:0/1616436427 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f155c15a370 0x7f155c0c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:38.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.595+0000 7f1561634640 1 --2- 172.21.15.69:0/1616436427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f155c1623c0 0x7f155c159ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:38.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.595+0000 7f1561634640 1 --2- 172.21.15.69:0/1616436427 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f155c0b91d0 0x7f155c0cd3b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:38.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.595+0000 7f1561634640 1 -- 172.21.15.69:0/1616436427 >> 172.21.15.69:0/1616436427 conn(0x7f155c0d3aa0 msgr2=0x7f155c071ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:38.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.595+0000 7f1561634640 1 -- 172.21.15.69:0/1616436427 shutdown_connections 2024-04-25T10:59:38.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.595+0000 7f1561634640 1 -- 172.21.15.69:0/1616436427 wait complete. 2024-04-25T10:59:38.608 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:59:06.422674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:59:16.727980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:59:17.938486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:59:38.608 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:59:38.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.720+0000 7fa602d8f640 1 Processor -- start 2024-04-25T10:59:38.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.720+0000 7fa602d8f640 1 -- start start 2024-04-25T10:59:38.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.721+0000 7fa602d8f640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fa5fc0b2e10 0x7fa5fc0ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:38.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.721+0000 7fa602d8f640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa5fc0d2800 0x7fa5fc0b5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:38.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.721+0000 7fa602d8f640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5fc0ceed0 con 0x7fa5fc0d23f0 2024-04-25T10:59:38.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.721+0000 7fa602d8f640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fa5fc0b9620 con 0x7fa5fc0b2e10 2024-04-25T10:59:38.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.721+0000 7fa602d8f640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fa5fc0ceca0 con 0x7fa5fc0d2800 2024-04-25T10:59:38.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.721+0000 7fa600b2b640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fa5fc0b2e10 0x7fa5fc0ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:38.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.721+0000 7fa5fbfff640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fa5fc0d23f0 0x7fa5fc0b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:44270/0 (socket says 172.21.15.69:44270) 2024-04-25T10:59:38.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.721+0000 7fa5fbfff640 1 -- 172.21.15.69:0/3747244854 learned_addr learned my addr 172.21.15.69:0/3747244854 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:38.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.721+0000 7fa600b2b640 1 --2- 172.21.15.69:0/3747244854 >> v2:172.21.15.69:3300/0 conn(0x7fa5fc0b2e10 0x7fa5fc0ce760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43560/0 (socket says 172.21.15.69:43560) 2024-04-25T10:59:38.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.721+0000 7fa60132c640 1 --2- 172.21.15.69:0/3747244854 >> v2:172.21.15.146:3300/0 conn(0x7fa5fc0d2800 0x7fa5fc0b5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:38.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.722+0000 7fa600b2b640 1 -- 172.21.15.69:0/3747244854 >> v2:172.21.15.146:3300/0 conn(0x7fa5fc0d2800 msgr2=0x7fa5fc0b5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:38.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.722+0000 7fa600b2b640 1 --2- 172.21.15.69:0/3747244854 >> v2:172.21.15.146:3300/0 conn(0x7fa5fc0d2800 0x7fa5fc0b5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:38.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.722+0000 7fa5fb7fe640 1 -- 172.21.15.69:0/3747244854 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3269197183 0 0) 0x7fa5fc0ceed0 con 0x7fa5fc0d23f0 2024-04-25T10:59:38.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.722+0000 7fa600b2b640 1 -- 172.21.15.69:0/3747244854 >> v1:172.21.15.69:6789/0 conn(0x7fa5fc0d23f0 legacy=0x7fa5fc0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:38.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.722+0000 7fa600b2b640 1 -- 172.21.15.69:0/3747244854 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5fc0cf1b0 con 0x7fa5fc0b2e10 2024-04-25T10:59:38.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.722+0000 7fa600b2b640 1 --2- 172.21.15.69:0/3747244854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5fc0b2e10 0x7fa5fc0ce760 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc071150 tx=0x7fa5ec02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d919f57e0a23df9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:38.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.722+0000 7fa5fb7fe640 1 -- 172.21.15.69:0/3747244854 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec037070 con 0x7fa5fc0b2e10 2024-04-25T10:59:38.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.723+0000 7fa5fb7fe640 1 -- 172.21.15.69:0/3747244854 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5ec032440 con 0x7fa5fc0b2e10 2024-04-25T10:59:38.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.723+0000 7fa5fb7fe640 1 -- 172.21.15.69:0/3747244854 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec032810 con 0x7fa5fc0b2e10 2024-04-25T10:59:38.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.723+0000 7fa602d8f640 1 -- 172.21.15.69:0/3747244854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5fc0b2e10 msgr2=0x7fa5fc0ce760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:38.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.723+0000 7fa602d8f640 1 --2- 172.21.15.69:0/3747244854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5fc0b2e10 0x7fa5fc0ce760 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc071150 tx=0x7fa5ec02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:59:38.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.723+0000 7fa602d8f640 1 -- 172.21.15.69:0/3747244854 shutdown_connections 2024-04-25T10:59:38.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.723+0000 7fa602d8f640 1 --2- 172.21.15.69:0/3747244854 >> v2:172.21.15.146:3300/0 conn(0x7fa5fc0d2800 0x7fa5fc0b5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:38.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.723+0000 7fa602d8f640 1 --2- 172.21.15.69:0/3747244854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5fc0b2e10 0x7fa5fc0ce760 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:38.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.723+0000 7fa602d8f640 1 -- 172.21.15.69:0/3747244854 >> 172.21.15.69:0/3747244854 conn(0x7fa5fc0d3aa0 msgr2=0x7fa5fc0d3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:38.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.723+0000 7fa602d8f640 1 -- 172.21.15.69:0/3747244854 shutdown_connections 2024-04-25T10:59:38.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.723+0000 7fa602d8f640 1 -- 172.21.15.69:0/3747244854 wait complete. 2024-04-25T10:59:38.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.725+0000 7fa602d8f640 1 Processor -- start 2024-04-25T10:59:38.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.725+0000 7fa602d8f640 1 -- start start 2024-04-25T10:59:38.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.726+0000 7fa602d8f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5fc0b2e10 0x7fa5fc15cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:38.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.727+0000 7fa602d8f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5fc0d23f0 0x7fa5fc0a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:38.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.727+0000 7fa602d8f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5fc0d2800 0x7fa5fc0c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:38.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.727+0000 7fa602d8f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa5fc0cfa50 con 0x7fa5fc0d23f0 2024-04-25T10:59:38.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.727+0000 7fa602d8f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa5fc0cf770 con 0x7fa5fc0d2800 2024-04-25T10:59:38.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.727+0000 7fa602d8f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa5fc0cff70 con 0x7fa5fc0b2e10 2024-04-25T10:59:38.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.727+0000 7fa5fbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5fc0d23f0 0x7fa5fc0a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:38.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.727+0000 7fa5fbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5fc0d23f0 0x7fa5fc0a0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43570/0 (socket says 172.21.15.69:43570) 2024-04-25T10:59:38.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.727+0000 7fa5fbfff640 1 -- 172.21.15.69:0/345025656 learned_addr learned my addr 172.21.15.69:0/345025656 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:38.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.727+0000 7fa600b2b640 1 --2- 172.21.15.69:0/345025656 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5fc0b2e10 0x7fa5fc15cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:38.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.727+0000 7fa60132c640 1 --2- 172.21.15.69:0/345025656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5fc0d2800 0x7fa5fc0c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:38.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.727+0000 7fa5fbfff640 1 -- 172.21.15.69:0/345025656 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5fc0b2e10 msgr2=0x7fa5fc15cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:38.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.727+0000 7fa5fbfff640 1 --2- 172.21.15.69:0/345025656 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5fc0b2e10 0x7fa5fc15cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:38.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.727+0000 7fa5fbfff640 1 -- 172.21.15.69:0/345025656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5fc0d2800 msgr2=0x7fa5fc0c8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:38.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.727+0000 7fa5fbfff640 1 --2- 172.21.15.69:0/345025656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5fc0d2800 0x7fa5fc0c8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:38.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.727+0000 7fa5fbfff640 1 -- 172.21.15.69:0/345025656 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5fc15de70 con 0x7fa5fc0d23f0 2024-04-25T10:59:38.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.727+0000 7fa5fbfff640 1 --2- 172.21.15.69:0/345025656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5fc0d23f0 0x7fa5fc0a0ff0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400c410 tx=0x7fa5e400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:38.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.728+0000 7fa5f97fa640 1 -- 172.21.15.69:0/345025656 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e40074e0 con 0x7fa5fc0d23f0 2024-04-25T10:59:38.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.728+0000 7fa5f97fa640 1 -- 172.21.15.69:0/345025656 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5e40047e0 con 0x7fa5fc0d23f0 2024-04-25T10:59:38.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.728+0000 7fa602d8f640 1 -- 172.21.15.69:0/345025656 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5fc15e150 con 0x7fa5fc0d23f0 2024-04-25T10:59:38.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.728+0000 7fa5f97fa640 1 -- 172.21.15.69:0/345025656 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4004c10 con 0x7fa5fc0d23f0 2024-04-25T10:59:38.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.728+0000 7fa602d8f640 1 -- 172.21.15.69:0/345025656 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5fc15e710 con 0x7fa5fc0d23f0 2024-04-25T10:59:38.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.729+0000 7fa602d8f640 1 -- 172.21.15.69:0/345025656 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5fc050740 con 0x7fa5fc0d23f0 2024-04-25T10:59:38.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.729+0000 7fa5f97fa640 1 -- 172.21.15.69:0/345025656 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa5e4011070 con 0x7fa5fc0d23f0 2024-04-25T10:59:38.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.730+0000 7fa5f97fa640 1 --2- 172.21.15.69:0/345025656 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa5dc0744e0 0x7fa5dc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:38.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.730+0000 7fa5f97fa640 1 -- 172.21.15.69:0/345025656 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa5e4094c20 con 0x7fa5fc0d23f0 2024-04-25T10:59:38.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.730+0000 7fa600b2b640 1 --2- 172.21.15.69:0/345025656 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa5dc0744e0 0x7fa5dc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:38.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.730+0000 7fa600b2b640 1 --2- 172.21.15.69:0/345025656 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa5dc0744e0 0x7fa5dc0769f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec034000 tx=0x7fa5ec02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:38.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.734+0000 7fa5f97fa640 1 -- 172.21.15.69:0/345025656 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa5e405ed60 con 0x7fa5fc0d23f0 2024-04-25T10:59:38.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.910+0000 7fa602d8f640 1 -- 172.21.15.69:0/345025656 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa5fc0d4700 con 0x7fa5fc0d23f0 2024-04-25T10:59:38.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.911+0000 7fa5f97fa640 1 -- 172.21.15.69:0/345025656 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fa5e4067430 con 0x7fa5fc0d23f0 2024-04-25T10:59:38.911 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-25T10:59:38.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.913+0000 7fa602d8f640 1 -- 172.21.15.69:0/345025656 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa5dc0744e0 msgr2=0x7fa5dc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:38.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.913+0000 7fa602d8f640 1 --2- 172.21.15.69:0/345025656 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa5dc0744e0 0x7fa5dc0769f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec034000 tx=0x7fa5ec02e000 comp rx=0 tx=0).stop 2024-04-25T10:59:38.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.913+0000 7fa602d8f640 1 -- 172.21.15.69:0/345025656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5fc0d23f0 msgr2=0x7fa5fc0a0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:38.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.913+0000 7fa602d8f640 1 --2- 172.21.15.69:0/345025656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5fc0d23f0 0x7fa5fc0a0ff0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400c410 tx=0x7fa5e400c8d0 comp rx=0 tx=0).stop 2024-04-25T10:59:38.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.913+0000 7fa602d8f640 1 -- 172.21.15.69:0/345025656 shutdown_connections 2024-04-25T10:59:38.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.913+0000 7fa602d8f640 1 --2- 172.21.15.69:0/345025656 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa5dc0744e0 0x7fa5dc0769f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:38.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.913+0000 7fa602d8f640 1 --2- 172.21.15.69:0/345025656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5fc0d2800 0x7fa5fc0c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:38.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.913+0000 7fa602d8f640 1 --2- 172.21.15.69:0/345025656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5fc0d23f0 0x7fa5fc0a0ff0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:38.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.913+0000 7fa602d8f640 1 --2- 172.21.15.69:0/345025656 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5fc0b2e10 0x7fa5fc15cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:38.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.913+0000 7fa602d8f640 1 -- 172.21.15.69:0/345025656 >> 172.21.15.69:0/345025656 conn(0x7fa5fc0d3aa0 msgr2=0x7fa5fc0b66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:38.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.913+0000 7fa602d8f640 1 -- 172.21.15.69:0/345025656 shutdown_connections 2024-04-25T10:59:38.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:38.913+0000 7fa602d8f640 1 -- 172.21.15.69:0/345025656 wait complete. 2024-04-25T10:59:38.925 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:59:39.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.034+0000 7f17151fe640 1 Processor -- start 2024-04-25T10:59:39.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.034+0000 7f17151fe640 1 -- start start 2024-04-25T10:59:39.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.035+0000 7f17151fe640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f17100d1fc0 0x7f17100b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:39.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.035+0000 7f17151fe640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1710160c80 0x7f1710161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:39.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.035+0000 7f17151fe640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17100d03e0 con 0x7f17100d2fb0 2024-04-25T10:59:39.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.035+0000 7f17151fe640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f17100d2b20 con 0x7f17100d1fc0 2024-04-25T10:59:39.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.035+0000 7f17151fe640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f17100d23d0 con 0x7f1710160c80 2024-04-25T10:59:39.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.035+0000 7f170ed9d640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f17100d1fc0 0x7f17100b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:39.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.035+0000 7f170ed9d640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f17100d1fc0 0x7f17100b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43584/0 (socket says 172.21.15.69:43584) 2024-04-25T10:59:39.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.035+0000 7f170ed9d640 1 -- 172.21.15.69:0/3175668035 learned_addr learned my addr 172.21.15.69:0/3175668035 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:39.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.036+0000 7f1707fff640 1 --2- 172.21.15.69:0/3175668035 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1710160c80 0x7f1710161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:39.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.036+0000 7f170ed9d640 1 -- 172.21.15.69:0/3175668035 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1710160c80 msgr2=0x7f1710161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:39.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.036+0000 7f170ed9d640 1 --2- 172.21.15.69:0/3175668035 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1710160c80 0x7f1710161090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.036+0000 7f170ed9d640 1 -- 172.21.15.69:0/3175668035 >> v1:172.21.15.146:6789/0 conn(0x7f17100d2fb0 legacy=0x7f17100b77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:39.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.036+0000 7f170ed9d640 1 -- 172.21.15.69:0/3175668035 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17100cfb40 con 0x7f17100d1fc0 2024-04-25T10:59:39.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.036+0000 7f170ed9d640 1 --2- 172.21.15.69:0/3175668035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17100d1fc0 0x7f17100b3e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1700009870 tx=0x7f170002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90df298c12c7485 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:39.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.037+0000 7f170e59c640 1 -- 172.21.15.69:0/3175668035 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1700037070 con 0x7f17100d1fc0 2024-04-25T10:59:39.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.037+0000 7f170e59c640 1 -- 172.21.15.69:0/3175668035 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1700032440 con 0x7f17100d1fc0 2024-04-25T10:59:39.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.037+0000 7f170e59c640 1 -- 172.21.15.69:0/3175668035 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1700032810 con 0x7f17100d1fc0 2024-04-25T10:59:39.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.037+0000 7f17151fe640 1 -- 172.21.15.69:0/3175668035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17100d1fc0 msgr2=0x7f17100b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:39.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.037+0000 7f17151fe640 1 --2- 172.21.15.69:0/3175668035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17100d1fc0 0x7f17100b3e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1700009870 tx=0x7f170002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.037+0000 7f17151fe640 1 -- 172.21.15.69:0/3175668035 shutdown_connections 2024-04-25T10:59:39.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.037+0000 7f17151fe640 1 --2- 172.21.15.69:0/3175668035 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1710160c80 0x7f1710161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.037+0000 7f17151fe640 1 --2- 172.21.15.69:0/3175668035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17100d1fc0 0x7f17100b3e20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.037+0000 7f17151fe640 1 -- 172.21.15.69:0/3175668035 >> 172.21.15.69:0/3175668035 conn(0x7f171009b7d0 msgr2=0x7f171009bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:39.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.037+0000 7f17151fe640 1 -- 172.21.15.69:0/3175668035 shutdown_connections 2024-04-25T10:59:39.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.037+0000 7f17151fe640 1 -- 172.21.15.69:0/3175668035 wait complete. 2024-04-25T10:59:39.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.039+0000 7f17151fe640 1 Processor -- start 2024-04-25T10:59:39.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.039+0000 7f17151fe640 1 -- start start 2024-04-25T10:59:39.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.040+0000 7f17151fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17100d1fc0 0x7f171015fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:39.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.041+0000 7f17151fe640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17100d2fb0 0x7f171013a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:39.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.041+0000 7f17151fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1710160c80 0x7f171013a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:39.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.041+0000 7f17151fe640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f17100cf780 con 0x7f1710160c80 2024-04-25T10:59:39.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.041+0000 7f17151fe640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f17100d0100 con 0x7f17100d1fc0 2024-04-25T10:59:39.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.041+0000 7f17151fe640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f17100cf900 con 0x7f17100d2fb0 2024-04-25T10:59:39.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.041+0000 7f170f59e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1710160c80 0x7f171013a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:39.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.041+0000 7f170ed9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17100d1fc0 0x7f171015fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:39.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.041+0000 7f170f59e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1710160c80 0x7f171013a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43598/0 (socket says 172.21.15.69:43598) 2024-04-25T10:59:39.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.041+0000 7f1707fff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17100d2fb0 0x7f171013a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:39.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.041+0000 7f170f59e640 1 -- 172.21.15.69:0/1462345961 learned_addr learned my addr 172.21.15.69:0/1462345961 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:39.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.041+0000 7f170ed9d640 1 -- 172.21.15.69:0/1462345961 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17100d2fb0 msgr2=0x7f171013a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:39.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.041+0000 7f170ed9d640 1 --2- 172.21.15.69:0/1462345961 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17100d2fb0 0x7f171013a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.041+0000 7f170ed9d640 1 -- 172.21.15.69:0/1462345961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1710160c80 msgr2=0x7f171013a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:39.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.041+0000 7f170ed9d640 1 --2- 172.21.15.69:0/1462345961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1710160c80 0x7f171013a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.041+0000 7f170ed9d640 1 -- 172.21.15.69:0/1462345961 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f171014f890 con 0x7f17100d1fc0 2024-04-25T10:59:39.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.041+0000 7f170ed9d640 1 --2- 172.21.15.69:0/1462345961 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17100d1fc0 0x7f171015fae0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1700031000 tx=0x7f17000046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:39.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.041+0000 7f17077fe640 1 -- 172.21.15.69:0/1462345961 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1700041020 con 0x7f17100d1fc0 2024-04-25T10:59:39.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.041+0000 7f17151fe640 1 -- 172.21.15.69:0/1462345961 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f171013b770 con 0x7f17100d1fc0 2024-04-25T10:59:39.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.041+0000 7f17077fe640 1 -- 172.21.15.69:0/1462345961 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1700030660 con 0x7f17100d1fc0 2024-04-25T10:59:39.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.042+0000 7f17077fe640 1 -- 172.21.15.69:0/1462345961 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1700037040 con 0x7f17100d1fc0 2024-04-25T10:59:39.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.042+0000 7f17151fe640 1 -- 172.21.15.69:0/1462345961 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f171014fb50 con 0x7f17100d1fc0 2024-04-25T10:59:39.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.043+0000 7f17151fe640 1 -- 172.21.15.69:0/1462345961 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17100cf560 con 0x7f17100d1fc0 2024-04-25T10:59:39.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.043+0000 7f17077fe640 1 -- 172.21.15.69:0/1462345961 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1700044020 con 0x7f17100d1fc0 2024-04-25T10:59:39.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.044+0000 7f17077fe640 1 --2- 172.21.15.69:0/1462345961 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f16f00744b0 0x7f16f00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:39.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.044+0000 7f17077fe640 1 -- 172.21.15.69:0/1462345961 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f17000b57e0 con 0x7f17100d1fc0 2024-04-25T10:59:39.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.044+0000 7f1707fff640 1 --2- 172.21.15.69:0/1462345961 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f16f00744b0 0x7f16f00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:39.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.044+0000 7f1707fff640 1 --2- 172.21.15.69:0/1462345961 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f16f00744b0 0x7f16f00769c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f16f8010ce0 tx=0x7f16f8012040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:39.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.053+0000 7f17077fe640 1 -- 172.21.15.69:0/1462345961 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f170004b030 con 0x7f17100d1fc0 2024-04-25T10:59:39.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.227+0000 7f17151fe640 1 -- 172.21.15.69:0/1462345961 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f171013be30 con 0x7f17100d1fc0 2024-04-25T10:59:39.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.228+0000 7f17077fe640 1 -- 172.21.15.69:0/1462345961 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f170007f920 con 0x7f17100d1fc0 2024-04-25T10:59:39.228 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-25T10:59:39.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.230+0000 7f17151fe640 1 -- 172.21.15.69:0/1462345961 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f16f00744b0 msgr2=0x7f16f00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:39.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.230+0000 7f17151fe640 1 --2- 172.21.15.69:0/1462345961 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f16f00744b0 0x7f16f00769c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f16f8010ce0 tx=0x7f16f8012040 comp rx=0 tx=0).stop 2024-04-25T10:59:39.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.230+0000 7f17151fe640 1 -- 172.21.15.69:0/1462345961 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17100d1fc0 msgr2=0x7f171015fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:39.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.230+0000 7f17151fe640 1 --2- 172.21.15.69:0/1462345961 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17100d1fc0 0x7f171015fae0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1700031000 tx=0x7f17000046c0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.230+0000 7f17151fe640 1 -- 172.21.15.69:0/1462345961 shutdown_connections 2024-04-25T10:59:39.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.230+0000 7f17151fe640 1 --2- 172.21.15.69:0/1462345961 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f16f00744b0 0x7f16f00769c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.230+0000 7f17151fe640 1 --2- 172.21.15.69:0/1462345961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1710160c80 0x7f171013a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.230+0000 7f17151fe640 1 --2- 172.21.15.69:0/1462345961 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17100d2fb0 0x7f171013a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.230+0000 7f17151fe640 1 --2- 172.21.15.69:0/1462345961 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17100d1fc0 0x7f171015fae0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.230+0000 7f17151fe640 1 -- 172.21.15.69:0/1462345961 >> 172.21.15.69:0/1462345961 conn(0x7f171009b7d0 msgr2=0x7f17100cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:39.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.230+0000 7f17151fe640 1 -- 172.21.15.69:0/1462345961 shutdown_connections 2024-04-25T10:59:39.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.230+0000 7f17151fe640 1 -- 172.21.15.69:0/1462345961 wait complete. 2024-04-25T10:59:39.241 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:59:39.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.351+0000 7f019eabd640 1 Processor -- start 2024-04-25T10:59:39.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.351+0000 7f019eabd640 1 -- start start 2024-04-25T10:59:39.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.353+0000 7f019eabd640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0198070820 0x7f019815f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:39.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.353+0000 7f019eabd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f019815fe40 0x7f019806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:39.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.353+0000 7f019eabd640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0198070c30 con 0x7f0198158850 2024-04-25T10:59:39.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.353+0000 7f019eabd640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f0198158c60 con 0x7f0198070820 2024-04-25T10:59:39.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.353+0000 7f019eabd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f019815ef90 con 0x7f019815fe40 2024-04-25T10:59:39.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.353+0000 7f0197fff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0198070820 0x7f019815f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:39.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.353+0000 7f019c859640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f0198158850 0x7f019815e780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:44284/0 (socket says 172.21.15.69:44284) 2024-04-25T10:59:39.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.353+0000 7f019c859640 1 -- 172.21.15.69:0/4250088042 learned_addr learned my addr 172.21.15.69:0/4250088042 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:39.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.353+0000 7f0197fff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0198070820 0x7f019815f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43614/0 (socket says 172.21.15.69:43614) 2024-04-25T10:59:39.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.353+0000 7f019d05a640 1 --2- 172.21.15.69:0/4250088042 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f019815fe40 0x7f019806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:39.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.353+0000 7f0197fff640 1 -- 172.21.15.69:0/4250088042 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f019815fe40 msgr2=0x7f019806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:39.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.353+0000 7f0197fff640 1 --2- 172.21.15.69:0/4250088042 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f019815fe40 0x7f019806d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.353+0000 7f0197fff640 1 -- 172.21.15.69:0/4250088042 >> v1:172.21.15.69:6789/0 conn(0x7f0198158850 legacy=0x7f019815e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:39.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.353+0000 7f0197fff640 1 -- 172.21.15.69:0/4250088042 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f019815f270 con 0x7f0198070820 2024-04-25T10:59:39.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.353+0000 7f01977fe640 1 -- 172.21.15.69:0/4250088042 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1181087632 0 0) 0x7f0198070c30 con 0x7f0198158850 2024-04-25T10:59:39.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.354+0000 7f0197fff640 1 --2- 172.21.15.69:0/4250088042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0198070820 0x7f019815f900 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f018c0097a0 tx=0x7f018c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4747448512ab2d28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:39.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.354+0000 7f01977fe640 1 -- 172.21.15.69:0/4250088042 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c037070 con 0x7f0198070820 2024-04-25T10:59:39.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.354+0000 7f01977fe640 1 -- 172.21.15.69:0/4250088042 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f018c032440 con 0x7f0198070820 2024-04-25T10:59:39.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.354+0000 7f01977fe640 1 -- 172.21.15.69:0/4250088042 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c032810 con 0x7f0198070820 2024-04-25T10:59:39.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.354+0000 7f019eabd640 1 -- 172.21.15.69:0/4250088042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0198070820 msgr2=0x7f019815f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:39.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.354+0000 7f019eabd640 1 --2- 172.21.15.69:0/4250088042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0198070820 0x7f019815f900 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f018c0097a0 tx=0x7f018c02a180 comp rx=0 tx=0).stop 2024-04-25T10:59:39.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.355+0000 7f019eabd640 1 -- 172.21.15.69:0/4250088042 shutdown_connections 2024-04-25T10:59:39.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.355+0000 7f019eabd640 1 --2- 172.21.15.69:0/4250088042 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f019815fe40 0x7f019806d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.355+0000 7f019eabd640 1 --2- 172.21.15.69:0/4250088042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0198070820 0x7f019815f900 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.355+0000 7f019eabd640 1 -- 172.21.15.69:0/4250088042 >> 172.21.15.69:0/4250088042 conn(0x7f01980730b0 msgr2=0x7f01980734c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:39.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.355+0000 7f019eabd640 1 -- 172.21.15.69:0/4250088042 shutdown_connections 2024-04-25T10:59:39.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.355+0000 7f019eabd640 1 -- 172.21.15.69:0/4250088042 wait complete. 2024-04-25T10:59:39.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.356+0000 7f019eabd640 1 Processor -- start 2024-04-25T10:59:39.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.356+0000 7f019eabd640 1 -- start start 2024-04-25T10:59:39.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.358+0000 7f019eabd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0198070820 0x7f0198157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:39.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.358+0000 7f019eabd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0198158850 0x7f0198143f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:39.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.358+0000 7f019eabd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f019815fe40 0x7f01981f4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:39.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.358+0000 7f019eabd640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f019806db10 con 0x7f0198158850 2024-04-25T10:59:39.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.358+0000 7f019eabd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f019806f040 con 0x7f0198070820 2024-04-25T10:59:39.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.358+0000 7f019eabd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f019806dc90 con 0x7f019815fe40 2024-04-25T10:59:39.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.358+0000 7f0197fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0198158850 0x7f0198143f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:39.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.358+0000 7f0197fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0198158850 0x7f0198143f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43630/0 (socket says 172.21.15.69:43630) 2024-04-25T10:59:39.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.358+0000 7f0197fff640 1 -- 172.21.15.69:0/569950846 learned_addr learned my addr 172.21.15.69:0/569950846 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:39.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.358+0000 7f019c859640 1 --2- 172.21.15.69:0/569950846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0198070820 0x7f0198157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:39.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.358+0000 7f019d05a640 1 --2- 172.21.15.69:0/569950846 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f019815fe40 0x7f01981f4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:39.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.358+0000 7f0197fff640 1 -- 172.21.15.69:0/569950846 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f019815fe40 msgr2=0x7f01981f4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:39.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.358+0000 7f0197fff640 1 --2- 172.21.15.69:0/569950846 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f019815fe40 0x7f01981f4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.358+0000 7f0197fff640 1 -- 172.21.15.69:0/569950846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0198070820 msgr2=0x7f0198157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:39.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.358+0000 7f0197fff640 1 --2- 172.21.15.69:0/569950846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0198070820 0x7f0198157b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.358+0000 7f0197fff640 1 -- 172.21.15.69:0/569950846 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f019815a610 con 0x7f0198158850 2024-04-25T10:59:39.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.358+0000 7f019c859640 1 --2- 172.21.15.69:0/569950846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0198070820 0x7f0198157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:39.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.358+0000 7f0197fff640 1 --2- 172.21.15.69:0/569950846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0198158850 0x7f0198143f00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f018c035040 tx=0x7f018c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:39.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.359+0000 7f01957fa640 1 -- 172.21.15.69:0/569950846 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c037050 con 0x7f0198158850 2024-04-25T10:59:39.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.359+0000 7f01957fa640 1 -- 172.21.15.69:0/569950846 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f018c038040 con 0x7f0198158850 2024-04-25T10:59:39.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.359+0000 7f019eabd640 1 -- 172.21.15.69:0/569950846 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f019815a8f0 con 0x7f0198158850 2024-04-25T10:59:39.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.359+0000 7f01957fa640 1 -- 172.21.15.69:0/569950846 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c004490 con 0x7f0198158850 2024-04-25T10:59:39.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.359+0000 7f019eabd640 1 -- 172.21.15.69:0/569950846 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01981f4820 con 0x7f0198158850 2024-04-25T10:59:39.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.360+0000 7f019eabd640 1 -- 172.21.15.69:0/569950846 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01980761c0 con 0x7f0198158850 2024-04-25T10:59:39.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.360+0000 7f01957fa640 1 -- 172.21.15.69:0/569950846 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f018c030900 con 0x7f0198158850 2024-04-25T10:59:39.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.361+0000 7f01957fa640 1 --2- 172.21.15.69:0/569950846 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f01840744b0 0x7f01840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:39.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.361+0000 7f019c859640 1 --2- 172.21.15.69:0/569950846 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f01840744b0 0x7f01840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:39.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.361+0000 7f01957fa640 1 -- 172.21.15.69:0/569950846 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f018c0b5c50 con 0x7f0198158850 2024-04-25T10:59:39.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.361+0000 7f019c859640 1 --2- 172.21.15.69:0/569950846 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f01840744b0 0x7f01840769c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0188000f60 tx=0x7f0188007ca0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:39.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.366+0000 7f01957fa640 1 -- 172.21.15.69:0/569950846 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f018c0bb020 con 0x7f0198158850 2024-04-25T10:59:39.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.541+0000 7f019eabd640 1 -- 172.21.15.69:0/569950846 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0198050740 con 0x7f0198158850 2024-04-25T10:59:39.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.541+0000 7f01957fa640 1 -- 172.21.15.69:0/569950846 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f018c080470 con 0x7f0198158850 2024-04-25T10:59:39.542 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-25T10:59:39.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.543+0000 7f019eabd640 1 -- 172.21.15.69:0/569950846 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f01840744b0 msgr2=0x7f01840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:39.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.543+0000 7f019eabd640 1 --2- 172.21.15.69:0/569950846 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f01840744b0 0x7f01840769c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0188000f60 tx=0x7f0188007ca0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.543+0000 7f019eabd640 1 -- 172.21.15.69:0/569950846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0198158850 msgr2=0x7f0198143f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:39.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.544+0000 7f019eabd640 1 --2- 172.21.15.69:0/569950846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0198158850 0x7f0198143f00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f018c035040 tx=0x7f018c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.544+0000 7f019eabd640 1 -- 172.21.15.69:0/569950846 shutdown_connections 2024-04-25T10:59:39.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.544+0000 7f019eabd640 1 --2- 172.21.15.69:0/569950846 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f01840744b0 0x7f01840769c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.544+0000 7f019eabd640 1 --2- 172.21.15.69:0/569950846 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f019815fe40 0x7f01981f4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.544+0000 7f019eabd640 1 --2- 172.21.15.69:0/569950846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0198158850 0x7f0198143f00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.544+0000 7f019eabd640 1 --2- 172.21.15.69:0/569950846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0198070820 0x7f0198157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.544+0000 7f019eabd640 1 -- 172.21.15.69:0/569950846 >> 172.21.15.69:0/569950846 conn(0x7f01980730b0 msgr2=0x7f01980684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:39.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.544+0000 7f019eabd640 1 -- 172.21.15.69:0/569950846 shutdown_connections 2024-04-25T10:59:39.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.544+0000 7f019eabd640 1 -- 172.21.15.69:0/569950846 wait complete. 2024-04-25T10:59:39.556 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:59:39.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.662+0000 7f19c21d7640 1 Processor -- start 2024-04-25T10:59:39.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.662+0000 7f19c21d7640 1 -- start start 2024-04-25T10:59:39.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.664+0000 7f19c21d7640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f19bc0b8290 0x7f19bc160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:39.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.664+0000 7f19c21d7640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f19bc15df80 0x7f19bc160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:39.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.664+0000 7f19c21d7640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19bc1615f0 con 0x7f19bc164b00 2024-04-25T10:59:39.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.664+0000 7f19c21d7640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f19bc164670 con 0x7f19bc15df80 2024-04-25T10:59:39.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.664+0000 7f19c21d7640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f19bc164020 con 0x7f19bc0b8290 2024-04-25T10:59:39.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.664+0000 7f19bbfff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f19bc15df80 0x7f19bc160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:39.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.664+0000 7f19bb7fe640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f19bc164b00 0x7f19bc0b7b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:44288/0 (socket says 172.21.15.69:44288) 2024-04-25T10:59:39.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.664+0000 7f19bb7fe640 1 -- 172.21.15.69:0/3364957365 learned_addr learned my addr 172.21.15.69:0/3364957365 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:39.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.664+0000 7f19baffd640 1 --2- 172.21.15.69:0/3364957365 >> v2:172.21.15.146:3300/0 conn(0x7f19bc0b8290 0x7f19bc160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:39.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.664+0000 7f19bbfff640 1 -- 172.21.15.69:0/3364957365 >> v2:172.21.15.146:3300/0 conn(0x7f19bc0b8290 msgr2=0x7f19bc160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:39.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.664+0000 7f19bbfff640 1 --2- 172.21.15.69:0/3364957365 >> v2:172.21.15.146:3300/0 conn(0x7f19bc0b8290 0x7f19bc160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.664+0000 7f19bbfff640 1 -- 172.21.15.69:0/3364957365 >> v1:172.21.15.69:6789/0 conn(0x7f19bc164b00 legacy=0x7f19bc0b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:39.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.664+0000 7f19bbfff640 1 -- 172.21.15.69:0/3364957365 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19bc164300 con 0x7f19bc15df80 2024-04-25T10:59:39.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.664+0000 7f19ba7fc640 1 -- 172.21.15.69:0/3364957365 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3421207438 0 0) 0x7f19bc1615f0 con 0x7f19bc164b00 2024-04-25T10:59:39.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.665+0000 7f19bbfff640 1 --2- 172.21.15.69:0/3364957365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19bc15df80 0x7f19bc160dd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f19bc0d2540 tx=0x7f19b402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a90be7c3f78e1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:39.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.665+0000 7f19ba7fc640 1 -- 172.21.15.69:0/3364957365 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b4007740 con 0x7f19bc15df80 2024-04-25T10:59:39.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.665+0000 7f19ba7fc640 1 -- 172.21.15.69:0/3364957365 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19b40078e0 con 0x7f19bc15df80 2024-04-25T10:59:39.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.665+0000 7f19ba7fc640 1 -- 172.21.15.69:0/3364957365 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b4007cb0 con 0x7f19bc15df80 2024-04-25T10:59:39.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.665+0000 7f19c21d7640 1 -- 172.21.15.69:0/3364957365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19bc15df80 msgr2=0x7f19bc160dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:39.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.665+0000 7f19c21d7640 1 --2- 172.21.15.69:0/3364957365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19bc15df80 0x7f19bc160dd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f19bc0d2540 tx=0x7f19b402a680 comp rx=0 tx=0).stop 2024-04-25T10:59:39.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.666+0000 7f19c21d7640 1 -- 172.21.15.69:0/3364957365 shutdown_connections 2024-04-25T10:59:39.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.666+0000 7f19c21d7640 1 --2- 172.21.15.69:0/3364957365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19bc15df80 0x7f19bc160dd0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.666+0000 7f19c21d7640 1 --2- 172.21.15.69:0/3364957365 >> v2:172.21.15.146:3300/0 conn(0x7f19bc0b8290 0x7f19bc160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.666+0000 7f19c21d7640 1 -- 172.21.15.69:0/3364957365 >> 172.21.15.69:0/3364957365 conn(0x7f19bc0d2bc0 msgr2=0x7f19bc0d2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:39.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.666+0000 7f19c21d7640 1 -- 172.21.15.69:0/3364957365 shutdown_connections 2024-04-25T10:59:39.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.666+0000 7f19c21d7640 1 -- 172.21.15.69:0/3364957365 wait complete. 2024-04-25T10:59:39.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.667+0000 7f19c21d7640 1 Processor -- start 2024-04-25T10:59:39.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.668+0000 7f19c21d7640 1 -- start start 2024-04-25T10:59:39.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.669+0000 7f19c21d7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19bc15df80 0x7f19bc0c9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:39.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.669+0000 7f19c21d7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19bc164b00 0x7f19bc0cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:39.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.669+0000 7f19c21d7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19bc0ca960 0x7f19bc14d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:39.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.669+0000 7f19c21d7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f19bc0b4a00 con 0x7f19bc0ca960 2024-04-25T10:59:39.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.669+0000 7f19c21d7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f19bc0b4880 con 0x7f19bc15df80 2024-04-25T10:59:39.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.669+0000 7f19c21d7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f19bc0b4b80 con 0x7f19bc164b00 2024-04-25T10:59:39.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.669+0000 7f19bbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19bc0ca960 0x7f19bc14d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:39.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.669+0000 7f19bbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19bc0ca960 0x7f19bc14d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43650/0 (socket says 172.21.15.69:43650) 2024-04-25T10:59:39.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.669+0000 7f19bbfff640 1 -- 172.21.15.69:0/3097024428 learned_addr learned my addr 172.21.15.69:0/3097024428 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:39.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.669+0000 7f19baffd640 1 --2- 172.21.15.69:0/3097024428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19bc164b00 0x7f19bc0cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:39.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.669+0000 7f19bb7fe640 1 --2- 172.21.15.69:0/3097024428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19bc15df80 0x7f19bc0c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:39.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.669+0000 7f19bbfff640 1 -- 172.21.15.69:0/3097024428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19bc164b00 msgr2=0x7f19bc0cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:39.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.669+0000 7f19bbfff640 1 --2- 172.21.15.69:0/3097024428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19bc164b00 0x7f19bc0cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.669+0000 7f19bbfff640 1 -- 172.21.15.69:0/3097024428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19bc15df80 msgr2=0x7f19bc0c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:39.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.669+0000 7f19bbfff640 1 --2- 172.21.15.69:0/3097024428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19bc15df80 0x7f19bc0c9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.669+0000 7f19bbfff640 1 -- 172.21.15.69:0/3097024428 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19bc0cb5d0 con 0x7f19bc0ca960 2024-04-25T10:59:39.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.669+0000 7f19bbfff640 1 --2- 172.21.15.69:0/3097024428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19bc0ca960 0x7f19bc14d950 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f19b4009e50 tx=0x7f19b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:39.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.670+0000 7f19a3fff640 1 -- 172.21.15.69:0/3097024428 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b403f070 con 0x7f19bc0ca960 2024-04-25T10:59:39.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.670+0000 7f19a3fff640 1 -- 172.21.15.69:0/3097024428 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19b4031bf0 con 0x7f19bc0ca960 2024-04-25T10:59:39.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.670+0000 7f19c21d7640 1 -- 172.21.15.69:0/3097024428 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19bc0cb8b0 con 0x7f19bc0ca960 2024-04-25T10:59:39.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.670+0000 7f19a3fff640 1 -- 172.21.15.69:0/3097024428 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b4009450 con 0x7f19bc0ca960 2024-04-25T10:59:39.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.670+0000 7f19c21d7640 1 -- 172.21.15.69:0/3097024428 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19bc0cbe70 con 0x7f19bc0ca960 2024-04-25T10:59:39.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.671+0000 7f19c21d7640 1 -- 172.21.15.69:0/3097024428 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19bc050740 con 0x7f19bc0ca960 2024-04-25T10:59:39.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.671+0000 7f19a3fff640 1 -- 172.21.15.69:0/3097024428 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f19b403b410 con 0x7f19bc0ca960 2024-04-25T10:59:39.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.672+0000 7f19a3fff640 1 --2- 172.21.15.69:0/3097024428 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f19a80744e0 0x7f19a80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:39.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.672+0000 7f19a3fff640 1 -- 172.21.15.69:0/3097024428 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f19b40b68d0 con 0x7f19bc0ca960 2024-04-25T10:59:39.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.672+0000 7f19bb7fe640 1 --2- 172.21.15.69:0/3097024428 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f19a80744e0 0x7f19a80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:39.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.672+0000 7f19bb7fe640 1 --2- 172.21.15.69:0/3097024428 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f19a80744e0 0x7f19a80769f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f19ac007fd0 tx=0x7f19ac007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:39.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.676+0000 7f19a3fff640 1 -- 172.21.15.69:0/3097024428 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f19b4080a10 con 0x7f19bc0ca960 2024-04-25T10:59:39.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.853+0000 7f19c21d7640 1 -- 172.21.15.69:0/3097024428 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f19bc0d4170 con 0x7f19bc0ca960 2024-04-25T10:59:39.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.854+0000 7f19a3fff640 1 -- 172.21.15.69:0/3097024428 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f19b4084d10 con 0x7f19bc0ca960 2024-04-25T10:59:39.855 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:39.855 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":20,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T10:59:20.630151+0000","last_up_change":"2024-04-25T10:59:05.546839+0000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:59:06.422674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:59:16.727980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:59:17.938486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6825","nonce":1876101065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6827","nonce":1876101065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6831","nonce":1876101065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6829","nonce":1876101065}]},"public_addr":"172.21.15.69:6825/1876101065","cluster_addr":"172.21.15.69:6827/1876101065","heartbeat_back_addr":"172.21.15.69:6831/1876101065","heartbeat_front_addr":"172.21.15.69:6829/1876101065","state":["exists","up"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6801","nonce":1806438563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6803","nonce":1806438563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6807","nonce":1806438563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6805","nonce":1806438563}]},"public_addr":"172.21.15.69:6801/1806438563","cluster_addr":"172.21.15.69:6803/1806438563","heartbeat_back_addr":"172.21.15.69:6807/1806438563","heartbeat_front_addr":"172.21.15.69:6805/1806438563","state":["exists","up"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6817","nonce":1593617983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6819","nonce":1593617983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6823","nonce":1593617983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6821","nonce":1593617983}]},"public_addr":"172.21.15.69:6817/1593617983","cluster_addr":"172.21.15.69:6819/1593617983","heartbeat_back_addr":"172.21.15.69:6823/1593617983","heartbeat_front_addr":"172.21.15.69:6821/1593617983","state":["exists","up"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6809","nonce":3950661294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6811","nonce":3950661294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6815","nonce":3950661294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6813","nonce":3950661294}]},"public_addr":"172.21.15.69:6809/3950661294","cluster_addr":"172.21.15.69:6811/3950661294","heartbeat_back_addr":"172.21.15.69:6815/3950661294","heartbeat_front_addr":"172.21.15.69:6813/3950661294","state":["exists","up"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6801","nonce":2508929872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6803","nonce":2508929872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6807","nonce":2508929872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6805","nonce":2508929872}]},"public_addr":"172.21.15.146:6801/2508929872","cluster_addr":"172.21.15.146:6803/2508929872","heartbeat_back_addr":"172.21.15.146:6807/2508929872","heartbeat_front_addr":"172.21.15.146:6805/2508929872","state":["exists","up"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6811","nonce":1310911138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6816","nonce":1310911138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6823","nonce":1310911138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6821","nonce":1310911138}]},"public_addr":"172.21.15.146:6811/1310911138","cluster_addr":"172.21.15.146:6816/1310911138","heartbeat_back_addr":"172.21.15.146:6823/1310911138","heartbeat_front_addr":"172.21.15.146:6821/1310911138","state":["exists","up"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6810","nonce":634688021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6813","nonce":634688021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6820","nonce":634688021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6817","nonce":634688021}]},"public_addr":"172.21.15.146:6810/634688021","cluster_addr":"172.21.15.146:6813/634688021","heartbeat_back_addr":"172.21.15.146:6820/634688021","heartbeat_front_addr":"172.21.15.146:6817/634688021","state":["exists","up"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6825","nonce":3814215935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6827","nonce":3814215935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6831","nonce":3814215935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6829","nonce":3814215935}]},"public_addr":"172.21.15.146:6825/3814215935","cluster_addr":"172.21.15.146:6827/3814215935","heartbeat_back_addr":"172.21.15.146:6831/3814215935","heartbeat_front_addr":"172.21.15.146:6829/3814215935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.820200+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.850189+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.764560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.866248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.807897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.869881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.782310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.827131+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:59:39.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.856+0000 7f19c21d7640 1 -- 172.21.15.69:0/3097024428 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f19a80744e0 msgr2=0x7f19a80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:39.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.856+0000 7f19c21d7640 1 --2- 172.21.15.69:0/3097024428 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f19a80744e0 0x7f19a80769f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f19ac007fd0 tx=0x7f19ac007de0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.856+0000 7f19c21d7640 1 -- 172.21.15.69:0/3097024428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19bc0ca960 msgr2=0x7f19bc14d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:39.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.856+0000 7f19c21d7640 1 --2- 172.21.15.69:0/3097024428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19bc0ca960 0x7f19bc14d950 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f19b4009e50 tx=0x7f19b4004a40 comp rx=0 tx=0).stop 2024-04-25T10:59:39.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.856+0000 7f19c21d7640 1 -- 172.21.15.69:0/3097024428 shutdown_connections 2024-04-25T10:59:39.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.856+0000 7f19c21d7640 1 --2- 172.21.15.69:0/3097024428 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f19a80744e0 0x7f19a80769f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.856+0000 7f19c21d7640 1 --2- 172.21.15.69:0/3097024428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19bc0ca960 0x7f19bc14d950 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.856+0000 7f19c21d7640 1 --2- 172.21.15.69:0/3097024428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19bc164b00 0x7f19bc0cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.856+0000 7f19c21d7640 1 --2- 172.21.15.69:0/3097024428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19bc15df80 0x7f19bc0c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.856+0000 7f19c21d7640 1 -- 172.21.15.69:0/3097024428 >> 172.21.15.69:0/3097024428 conn(0x7f19bc0d2bc0 msgr2=0x7f19bc15e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:39.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.856+0000 7f19c21d7640 1 -- 172.21.15.69:0/3097024428 shutdown_connections 2024-04-25T10:59:39.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.856+0000 7f19c21d7640 1 -- 172.21.15.69:0/3097024428 wait complete. 2024-04-25T10:59:39.868 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:59:06.422674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:59:16.727980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:59:17.938486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:59:39.869 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:59:39.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.978+0000 7f31f4ef2640 1 Processor -- start 2024-04-25T10:59:39.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.978+0000 7f31f4ef2640 1 -- start start 2024-04-25T10:59:39.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.980+0000 7f31f4ef2640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f31f00d0e00 0x7f31f00d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:39.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.980+0000 7f31f4ef2640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f31f006c5c0 0x7f31f006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:39.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.980+0000 7f31f4ef2640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31f006cf10 con 0x7f31f00b7810 2024-04-25T10:59:39.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.980+0000 7f31f4ef2640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f31f00d5a60 con 0x7f31f00d0e00 2024-04-25T10:59:39.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.980+0000 7f31f4ef2640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f31f00d1850 con 0x7f31f006c5c0 2024-04-25T10:59:39.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.980+0000 7f31ee59c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f31f00d0e00 0x7f31f00d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:39.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.980+0000 7f31ee59c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f31f00d0e00 0x7f31f00d1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43662/0 (socket says 172.21.15.69:43662) 2024-04-25T10:59:39.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.980+0000 7f31ee59c640 1 -- 172.21.15.69:0/3525146525 learned_addr learned my addr 172.21.15.69:0/3525146525 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:39.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.980+0000 7f31edd9b640 1 --2- 172.21.15.69:0/3525146525 >> v2:172.21.15.146:3300/0 conn(0x7f31f006c5c0 0x7f31f006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:39.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.980+0000 7f31ee59c640 1 -- 172.21.15.69:0/3525146525 >> v2:172.21.15.146:3300/0 conn(0x7f31f006c5c0 msgr2=0x7f31f006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:39.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.980+0000 7f31ee59c640 1 --2- 172.21.15.69:0/3525146525 >> v2:172.21.15.146:3300/0 conn(0x7f31f006c5c0 0x7f31f006c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.980+0000 7f31ee59c640 1 -- 172.21.15.69:0/3525146525 >> v1:172.21.15.146:6789/0 conn(0x7f31f00b7810 legacy=0x7f31f00b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:39.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.980+0000 7f31ee59c640 1 -- 172.21.15.69:0/3525146525 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f0160de0 con 0x7f31f00d0e00 2024-04-25T10:59:39.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.980+0000 7f31edd9b640 1 --2- 172.21.15.69:0/3525146525 >> v2:172.21.15.146:3300/0 conn(0x7f31f006c5c0 0x7f31f006c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:39.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.981+0000 7f31ee59c640 1 --2- 172.21.15.69:0/3525146525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31f00d0e00 0x7f31f00d1210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f31f00d5be0 tx=0x7f31e002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae60e786fe7fc215 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:39.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.981+0000 7f31ed59a640 1 -- 172.21.15.69:0/3525146525 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e0037070 con 0x7f31f00d0e00 2024-04-25T10:59:39.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.981+0000 7f31ed59a640 1 -- 172.21.15.69:0/3525146525 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31e0032440 con 0x7f31f00d0e00 2024-04-25T10:59:39.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.981+0000 7f31ed59a640 1 -- 172.21.15.69:0/3525146525 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e0032810 con 0x7f31f00d0e00 2024-04-25T10:59:39.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.982+0000 7f31f4ef2640 1 -- 172.21.15.69:0/3525146525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31f00d0e00 msgr2=0x7f31f00d1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:39.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.982+0000 7f31f4ef2640 1 --2- 172.21.15.69:0/3525146525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31f00d0e00 0x7f31f00d1210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f31f00d5be0 tx=0x7f31e002a180 comp rx=0 tx=0).stop 2024-04-25T10:59:39.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.982+0000 7f31f4ef2640 1 -- 172.21.15.69:0/3525146525 shutdown_connections 2024-04-25T10:59:39.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.982+0000 7f31f4ef2640 1 --2- 172.21.15.69:0/3525146525 >> v2:172.21.15.146:3300/0 conn(0x7f31f006c5c0 0x7f31f006c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.982+0000 7f31f4ef2640 1 --2- 172.21.15.69:0/3525146525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31f00d0e00 0x7f31f00d1210 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.982+0000 7f31f4ef2640 1 -- 172.21.15.69:0/3525146525 >> 172.21.15.69:0/3525146525 conn(0x7f31f00d5d90 msgr2=0x7f31f00d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:39.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.982+0000 7f31f4ef2640 1 -- 172.21.15.69:0/3525146525 shutdown_connections 2024-04-25T10:59:39.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.982+0000 7f31f4ef2640 1 -- 172.21.15.69:0/3525146525 wait complete. 2024-04-25T10:59:39.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.984+0000 7f31f4ef2640 1 Processor -- start 2024-04-25T10:59:39.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.984+0000 7f31f4ef2640 1 -- start start 2024-04-25T10:59:39.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.985+0000 7f31f4ef2640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f31f006c5c0 0x7f31f00c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:39.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.985+0000 7f31f4ef2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31f00b7810 0x7f31f00ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:39.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.985+0000 7f31f4ef2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31f00d0e00 0x7f31f00acbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:39.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.985+0000 7f31f4ef2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f31f0160ad0 con 0x7f31f00b7810 2024-04-25T10:59:39.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.985+0000 7f31f4ef2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f31f01613a0 con 0x7f31f00d0e00 2024-04-25T10:59:39.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.985+0000 7f31f4ef2640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f31f00b5320 con 0x7f31f006c5c0 2024-04-25T10:59:39.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.985+0000 7f31edd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31f00b7810 0x7f31f00ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:39.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.985+0000 7f31edd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31f00b7810 0x7f31f00ac6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43670/0 (socket says 172.21.15.69:43670) 2024-04-25T10:59:39.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.985+0000 7f31edd9b640 1 -- 172.21.15.69:0/1685571663 learned_addr learned my addr 172.21.15.69:0/1685571663 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:39.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.985+0000 7f31eed9d640 1 --2- 172.21.15.69:0/1685571663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31f00d0e00 0x7f31f00acbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:39.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.985+0000 7f31ee59c640 1 --2- 172.21.15.69:0/1685571663 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f31f006c5c0 0x7f31f00c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:39.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.985+0000 7f31edd9b640 1 -- 172.21.15.69:0/1685571663 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f31f006c5c0 msgr2=0x7f31f00c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:39.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.985+0000 7f31edd9b640 1 --2- 172.21.15.69:0/1685571663 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f31f006c5c0 0x7f31f00c9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.985+0000 7f31edd9b640 1 -- 172.21.15.69:0/1685571663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31f00d0e00 msgr2=0x7f31f00acbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:39.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.985+0000 7f31edd9b640 1 --2- 172.21.15.69:0/1685571663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31f00d0e00 0x7f31f00acbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:39.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.985+0000 7f31edd9b640 1 -- 172.21.15.69:0/1685571663 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f00ca3e0 con 0x7f31f00b7810 2024-04-25T10:59:39.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.986+0000 7f31edd9b640 1 --2- 172.21.15.69:0/1685571663 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31f00b7810 0x7f31f00ac6a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f31e400c7f0 tx=0x7f31e400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:39.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.986+0000 7f31deffd640 1 -- 172.21.15.69:0/1685571663 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e4015070 con 0x7f31f00b7810 2024-04-25T10:59:39.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.986+0000 7f31deffd640 1 -- 172.21.15.69:0/1685571663 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31e4007820 con 0x7f31f00b7810 2024-04-25T10:59:39.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.986+0000 7f31f4ef2640 1 -- 172.21.15.69:0/1685571663 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31f00ca6c0 con 0x7f31f00b7810 2024-04-25T10:59:39.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.986+0000 7f31deffd640 1 -- 172.21.15.69:0/1685571663 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e4007bd0 con 0x7f31f00b7810 2024-04-25T10:59:39.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.986+0000 7f31f4ef2640 1 -- 172.21.15.69:0/1685571663 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31f00cac80 con 0x7f31f00b7810 2024-04-25T10:59:39.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.987+0000 7f31f4ef2640 1 -- 172.21.15.69:0/1685571663 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31f0050740 con 0x7f31f00b7810 2024-04-25T10:59:39.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.987+0000 7f31deffd640 1 -- 172.21.15.69:0/1685571663 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f31e4018070 con 0x7f31f00b7810 2024-04-25T10:59:39.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.988+0000 7f31deffd640 1 --2- 172.21.15.69:0/1685571663 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f31d00744e0 0x7f31d00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:39.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.988+0000 7f31deffd640 1 -- 172.21.15.69:0/1685571663 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f31e40952d0 con 0x7f31f00b7810 2024-04-25T10:59:39.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.988+0000 7f31ee59c640 1 --2- 172.21.15.69:0/1685571663 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f31d00744e0 0x7f31d00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:39.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.988+0000 7f31ee59c640 1 --2- 172.21.15.69:0/1685571663 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f31d00744e0 0x7f31d00769f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f31e0034000 tx=0x7f31e002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:39.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:39.992+0000 7f31deffd640 1 -- 172.21.15.69:0/1685571663 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f31e405f410 con 0x7f31f00b7810 2024-04-25T10:59:40.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.167+0000 7f31f4ef2640 1 -- 172.21.15.69:0/1685571663 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f31f00c86c0 con 0x7f31f00b7810 2024-04-25T10:59:40.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.168+0000 7f31deffd640 1 -- 172.21.15.69:0/1685571663 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f31e4063710 con 0x7f31f00b7810 2024-04-25T10:59:40.168 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-25T10:59:40.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.169+0000 7f31f4ef2640 1 -- 172.21.15.69:0/1685571663 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f31d00744e0 msgr2=0x7f31d00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:40.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.169+0000 7f31f4ef2640 1 --2- 172.21.15.69:0/1685571663 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f31d00744e0 0x7f31d00769f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f31e0034000 tx=0x7f31e002e000 comp rx=0 tx=0).stop 2024-04-25T10:59:40.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.169+0000 7f31f4ef2640 1 -- 172.21.15.69:0/1685571663 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31f00b7810 msgr2=0x7f31f00ac6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:40.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.169+0000 7f31f4ef2640 1 --2- 172.21.15.69:0/1685571663 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31f00b7810 0x7f31f00ac6a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f31e400c7f0 tx=0x7f31e400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:59:40.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.170+0000 7f31f4ef2640 1 -- 172.21.15.69:0/1685571663 shutdown_connections 2024-04-25T10:59:40.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.170+0000 7f31f4ef2640 1 --2- 172.21.15.69:0/1685571663 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f31d00744e0 0x7f31d00769f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:40.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.170+0000 7f31f4ef2640 1 --2- 172.21.15.69:0/1685571663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31f00d0e00 0x7f31f00acbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:40.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.170+0000 7f31f4ef2640 1 --2- 172.21.15.69:0/1685571663 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31f00b7810 0x7f31f00ac6a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:40.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.170+0000 7f31f4ef2640 1 --2- 172.21.15.69:0/1685571663 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f31f006c5c0 0x7f31f00c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:40.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.170+0000 7f31f4ef2640 1 -- 172.21.15.69:0/1685571663 >> 172.21.15.69:0/1685571663 conn(0x7f31f00d5d90 msgr2=0x7f31f0160230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:40.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.170+0000 7f31f4ef2640 1 -- 172.21.15.69:0/1685571663 shutdown_connections 2024-04-25T10:59:40.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.170+0000 7f31f4ef2640 1 -- 172.21.15.69:0/1685571663 wait complete. 2024-04-25T10:59:40.182 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:59:40.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.291+0000 7ff9e813f640 1 Processor -- start 2024-04-25T10:59:40.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.292+0000 7ff9e813f640 1 -- start start 2024-04-25T10:59:40.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.293+0000 7ff9e813f640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff9e00d0190 0x7ff9e00d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:40.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.293+0000 7ff9e813f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9e00d1f90 0x7ff9e00b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:40.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.293+0000 7ff9e813f640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9e00b3f30 con 0x7ff9e006ecd0 2024-04-25T10:59:40.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.293+0000 7ff9e813f640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7ff9e00b23e0 con 0x7ff9e00d0190 2024-04-25T10:59:40.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.293+0000 7ff9e813f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff9e00b4210 con 0x7ff9e00d1f90 2024-04-25T10:59:40.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.293+0000 7ff9e5edb640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff9e00d0190 0x7ff9e00d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:40.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.293+0000 7ff9e5edb640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff9e00d0190 0x7ff9e00d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43672/0 (socket says 172.21.15.69:43672) 2024-04-25T10:59:40.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.293+0000 7ff9e5edb640 1 -- 172.21.15.69:0/2658783403 learned_addr learned my addr 172.21.15.69:0/2658783403 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:40.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.293+0000 7ff9e56da640 1 --2- 172.21.15.69:0/2658783403 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9e00d1f90 0x7ff9e00b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:40.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.293+0000 7ff9e5edb640 1 -- 172.21.15.69:0/2658783403 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9e00d1f90 msgr2=0x7ff9e00b77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:40.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.293+0000 7ff9e5edb640 1 --2- 172.21.15.69:0/2658783403 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9e00d1f90 0x7ff9e00b77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:40.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.293+0000 7ff9e5edb640 1 -- 172.21.15.69:0/2658783403 >> v1:172.21.15.146:6789/0 conn(0x7ff9e006ecd0 legacy=0x7ff9e00b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:40.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.293+0000 7ff9e5edb640 1 -- 172.21.15.69:0/2658783403 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e00b2630 con 0x7ff9e00d0190 2024-04-25T10:59:40.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.294+0000 7ff9e5edb640 1 --2- 172.21.15.69:0/2658783403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9e00d0190 0x7ff9e00d1a50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8009e50 tx=0x7ff9d802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d80ac838f7967992 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:40.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.294+0000 7ff9e4ed9640 1 -- 172.21.15.69:0/2658783403 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8036070 con 0x7ff9e00d0190 2024-04-25T10:59:40.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.294+0000 7ff9e4ed9640 1 -- 172.21.15.69:0/2658783403 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9d8031600 con 0x7ff9e00d0190 2024-04-25T10:59:40.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.294+0000 7ff9e4ed9640 1 -- 172.21.15.69:0/2658783403 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d80319d0 con 0x7ff9e00d0190 2024-04-25T10:59:40.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.295+0000 7ff9e813f640 1 -- 172.21.15.69:0/2658783403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9e00d0190 msgr2=0x7ff9e00d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:40.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.295+0000 7ff9e813f640 1 --2- 172.21.15.69:0/2658783403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9e00d0190 0x7ff9e00d1a50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8009e50 tx=0x7ff9d802a5f0 comp rx=0 tx=0).stop 2024-04-25T10:59:40.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.295+0000 7ff9e813f640 1 -- 172.21.15.69:0/2658783403 shutdown_connections 2024-04-25T10:59:40.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.295+0000 7ff9e813f640 1 --2- 172.21.15.69:0/2658783403 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9e00d1f90 0x7ff9e00b77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:40.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.295+0000 7ff9e813f640 1 --2- 172.21.15.69:0/2658783403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9e00d0190 0x7ff9e00d1a50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:40.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.295+0000 7ff9e813f640 1 -- 172.21.15.69:0/2658783403 >> 172.21.15.69:0/2658783403 conn(0x7ff9e009b7d0 msgr2=0x7ff9e009bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:40.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.295+0000 7ff9e813f640 1 -- 172.21.15.69:0/2658783403 shutdown_connections 2024-04-25T10:59:40.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.295+0000 7ff9e813f640 1 -- 172.21.15.69:0/2658783403 wait complete. 2024-04-25T10:59:40.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.297+0000 7ff9e813f640 1 Processor -- start 2024-04-25T10:59:40.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.297+0000 7ff9e813f640 1 -- start start 2024-04-25T10:59:40.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.298+0000 7ff9e813f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9e006ecd0 0x7ff9e01e8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:40.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.298+0000 7ff9e813f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9e00d0190 0x7ff9e01e9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:40.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.298+0000 7ff9e813f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9e00d1f90 0x7ff9e00cae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:40.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.298+0000 7ff9e813f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e01607c0 con 0x7ff9e00d0190 2024-04-25T10:59:40.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.298+0000 7ff9e813f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e0160640 con 0x7ff9e00d1f90 2024-04-25T10:59:40.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.298+0000 7ff9e813f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff9e0160940 con 0x7ff9e006ecd0 2024-04-25T10:59:40.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.298+0000 7ff9e56da640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9e00d0190 0x7ff9e01e9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:40.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.298+0000 7ff9e56da640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9e00d0190 0x7ff9e01e9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43682/0 (socket says 172.21.15.69:43682) 2024-04-25T10:59:40.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.298+0000 7ff9e56da640 1 -- 172.21.15.69:0/3872681833 learned_addr learned my addr 172.21.15.69:0/3872681833 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:40.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.298+0000 7ff9e5edb640 1 --2- 172.21.15.69:0/3872681833 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9e006ecd0 0x7ff9e01e8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:40.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.298+0000 7ff9e66dc640 1 --2- 172.21.15.69:0/3872681833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9e00d1f90 0x7ff9e00cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:40.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.298+0000 7ff9e56da640 1 -- 172.21.15.69:0/3872681833 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9e006ecd0 msgr2=0x7ff9e01e8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:40.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.298+0000 7ff9e56da640 1 --2- 172.21.15.69:0/3872681833 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9e006ecd0 0x7ff9e01e8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:40.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.298+0000 7ff9e56da640 1 -- 172.21.15.69:0/3872681833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9e00d1f90 msgr2=0x7ff9e00cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:40.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.298+0000 7ff9e56da640 1 --2- 172.21.15.69:0/3872681833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9e00d1f90 0x7ff9e00cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:40.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.298+0000 7ff9e56da640 1 -- 172.21.15.69:0/3872681833 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e00b13f0 con 0x7ff9e00d0190 2024-04-25T10:59:40.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.298+0000 7ff9e66dc640 1 --2- 172.21.15.69:0/3872681833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9e00d1f90 0x7ff9e00cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:40.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.299+0000 7ff9e56da640 1 --2- 172.21.15.69:0/3872681833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9e00d0190 0x7ff9e01e9230 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000e8a0 tx=0x7ff9d000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:40.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.299+0000 7ff9d67fc640 1 -- 172.21.15.69:0/3872681833 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0017070 con 0x7ff9e00d0190 2024-04-25T10:59:40.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.299+0000 7ff9d67fc640 1 -- 172.21.15.69:0/3872681833 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9d000c940 con 0x7ff9e00d0190 2024-04-25T10:59:40.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.299+0000 7ff9e813f640 1 -- 172.21.15.69:0/3872681833 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9e006d170 con 0x7ff9e00d0190 2024-04-25T10:59:40.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.299+0000 7ff9d67fc640 1 -- 172.21.15.69:0/3872681833 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d000cd30 con 0x7ff9e00d0190 2024-04-25T10:59:40.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.299+0000 7ff9e813f640 1 -- 172.21.15.69:0/3872681833 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9e006d730 con 0x7ff9e00d0190 2024-04-25T10:59:40.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.300+0000 7ff9e813f640 1 -- 172.21.15.69:0/3872681833 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9e0050740 con 0x7ff9e00d0190 2024-04-25T10:59:40.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.300+0000 7ff9d67fc640 1 -- 172.21.15.69:0/3872681833 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff9d001a070 con 0x7ff9e00d0190 2024-04-25T10:59:40.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.301+0000 7ff9d67fc640 1 --2- 172.21.15.69:0/3872681833 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff9c40744b0 0x7ff9c40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:40.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.301+0000 7ff9e5edb640 1 --2- 172.21.15.69:0/3872681833 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff9c40744b0 0x7ff9c40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:40.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.301+0000 7ff9d67fc640 1 -- 172.21.15.69:0/3872681833 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff9d0095450 con 0x7ff9e00d0190 2024-04-25T10:59:40.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.301+0000 7ff9e5edb640 1 --2- 172.21.15.69:0/3872681833 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff9c40744b0 0x7ff9c40769c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8033000 tx=0x7ff9d8009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:40.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.305+0000 7ff9d67fc640 1 -- 172.21.15.69:0/3872681833 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff9d005f590 con 0x7ff9e00d0190 2024-04-25T10:59:40.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.479+0000 7ff9e813f640 1 -- 172.21.15.69:0/3872681833 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff9e00cf560 con 0x7ff9e00d0190 2024-04-25T10:59:40.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.480+0000 7ff9d67fc640 1 -- 172.21.15.69:0/3872681833 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7ff9d0063890 con 0x7ff9e00d0190 2024-04-25T10:59:40.481 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-25T10:59:40.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.482+0000 7ff9e813f640 1 -- 172.21.15.69:0/3872681833 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff9c40744b0 msgr2=0x7ff9c40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:40.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.482+0000 7ff9e813f640 1 --2- 172.21.15.69:0/3872681833 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff9c40744b0 0x7ff9c40769c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8033000 tx=0x7ff9d8009000 comp rx=0 tx=0).stop 2024-04-25T10:59:40.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.482+0000 7ff9e813f640 1 -- 172.21.15.69:0/3872681833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9e00d0190 msgr2=0x7ff9e01e9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:40.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.482+0000 7ff9e813f640 1 --2- 172.21.15.69:0/3872681833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9e00d0190 0x7ff9e01e9230 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000e8a0 tx=0x7ff9d000ed60 comp rx=0 tx=0).stop 2024-04-25T10:59:40.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.482+0000 7ff9e813f640 1 -- 172.21.15.69:0/3872681833 shutdown_connections 2024-04-25T10:59:40.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.482+0000 7ff9e813f640 1 --2- 172.21.15.69:0/3872681833 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff9c40744b0 0x7ff9c40769c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:40.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.482+0000 7ff9e813f640 1 --2- 172.21.15.69:0/3872681833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9e00d1f90 0x7ff9e00cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:40.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.482+0000 7ff9e813f640 1 --2- 172.21.15.69:0/3872681833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9e00d0190 0x7ff9e01e9230 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:40.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.482+0000 7ff9e813f640 1 --2- 172.21.15.69:0/3872681833 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9e006ecd0 0x7ff9e01e8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:40.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.482+0000 7ff9e813f640 1 -- 172.21.15.69:0/3872681833 >> 172.21.15.69:0/3872681833 conn(0x7ff9e009b7d0 msgr2=0x7ff9e00d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:40.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.482+0000 7ff9e813f640 1 -- 172.21.15.69:0/3872681833 shutdown_connections 2024-04-25T10:59:40.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.482+0000 7ff9e813f640 1 -- 172.21.15.69:0/3872681833 wait complete. 2024-04-25T10:59:40.494 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:59:40.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.603+0000 7f88ed4f9640 1 Processor -- start 2024-04-25T10:59:40.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.603+0000 7f88ed4f9640 1 -- start start 2024-04-25T10:59:40.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.605+0000 7f88ed4f9640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f88e8161710 0x7f88e815e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:40.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.605+0000 7f88ed4f9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88e8163fa0 0x7f88e81643b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:40.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.605+0000 7f88ed4f9640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88e81609b0 con 0x7f88e806c5c0 2024-04-25T10:59:40.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.605+0000 7f88ed4f9640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f88e8164cd0 con 0x7f88e8161710 2024-04-25T10:59:40.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.605+0000 7f88ed4f9640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f88e8160c30 con 0x7f88e8163fa0 2024-04-25T10:59:40.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.605+0000 7f88e77fe640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88e8163fa0 0x7f88e81643b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:40.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.605+0000 7f88e6ffd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f88e8161710 0x7f88e815e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:40.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.605+0000 7f88e6ffd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f88e8161710 0x7f88e815e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.69:49644/0 (socket says 172.21.15.69:49644) 2024-04-25T10:59:40.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.605+0000 7f88e77fe640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88e8163fa0 0x7f88e81643b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.69:54370/0 (socket says 172.21.15.69:54370) 2024-04-25T10:59:40.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.605+0000 7f88e6ffd640 1 -- 172.21.15.69:0/2000178311 learned_addr learned my addr 172.21.15.69:0/2000178311 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:40.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.605+0000 7f88e67fc640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f88e806c5c0 0x7f88e806c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.69:56608/0 (socket says 172.21.15.69:56608) 2024-04-25T10:59:40.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.606+0000 7f88e6ffd640 1 -- 172.21.15.69:0/2000178311 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88e8163fa0 msgr2=0x7f88e81643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:40.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.606+0000 7f88e6ffd640 1 --2- 172.21.15.69:0/2000178311 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88e8163fa0 0x7f88e81643b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:40.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.606+0000 7f88e6ffd640 1 -- 172.21.15.69:0/2000178311 >> v1:172.21.15.146:6789/0 conn(0x7f88e806c5c0 legacy=0x7f88e806c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:40.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.606+0000 7f88e6ffd640 1 -- 172.21.15.69:0/2000178311 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e8160f10 con 0x7f88e8161710 2024-04-25T10:59:40.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.606+0000 7f88e77fe640 1 --2- 172.21.15.69:0/2000178311 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88e8163fa0 0x7f88e81643b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:59:40.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.606+0000 7f88e6ffd640 1 --2- 172.21.15.69:0/2000178311 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88e8161710 0x7f88e815e080 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f88d8009e50 tx=0x7f88d802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb103703fed9d0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:40.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.607+0000 7f88e5ffb640 1 -- 172.21.15.69:0/2000178311 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d8036070 con 0x7f88e8161710 2024-04-25T10:59:40.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.607+0000 7f88e5ffb640 1 -- 172.21.15.69:0/2000178311 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88d8031600 con 0x7f88e8161710 2024-04-25T10:59:40.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.607+0000 7f88e5ffb640 1 -- 172.21.15.69:0/2000178311 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d80319d0 con 0x7f88e8161710 2024-04-25T10:59:40.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.607+0000 7f88ed4f9640 1 -- 172.21.15.69:0/2000178311 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88e8161710 msgr2=0x7f88e815e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:40.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.607+0000 7f88ed4f9640 1 --2- 172.21.15.69:0/2000178311 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88e8161710 0x7f88e815e080 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f88d8009e50 tx=0x7f88d802a680 comp rx=0 tx=0).stop 2024-04-25T10:59:40.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.607+0000 7f88ed4f9640 1 -- 172.21.15.69:0/2000178311 shutdown_connections 2024-04-25T10:59:40.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.607+0000 7f88ed4f9640 1 --2- 172.21.15.69:0/2000178311 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88e8163fa0 0x7f88e81643b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:40.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.607+0000 7f88ed4f9640 1 --2- 172.21.15.69:0/2000178311 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88e8161710 0x7f88e815e080 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:40.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.607+0000 7f88ed4f9640 1 -- 172.21.15.69:0/2000178311 >> 172.21.15.69:0/2000178311 conn(0x7f88e80d5d90 msgr2=0x7f88e80d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:40.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.607+0000 7f88ed4f9640 1 -- 172.21.15.69:0/2000178311 shutdown_connections 2024-04-25T10:59:40.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.607+0000 7f88ed4f9640 1 -- 172.21.15.69:0/2000178311 wait complete. 2024-04-25T10:59:40.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.609+0000 7f88ed4f9640 1 Processor -- start 2024-04-25T10:59:40.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.609+0000 7f88ed4f9640 1 -- start start 2024-04-25T10:59:40.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.610+0000 7f88ed4f9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88e806c5c0 0x7f88e81523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:40.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.610+0000 7f88ed4f9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88e8161710 0x7f88e80ce360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:40.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.610+0000 7f88ed4f9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88e8163fa0 0x7f88e80ba210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:40.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.611+0000 7f88ed4f9640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f88e8164a70 con 0x7f88e8161710 2024-04-25T10:59:40.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.611+0000 7f88ed4f9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f88e81648f0 con 0x7f88e8163fa0 2024-04-25T10:59:40.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.611+0000 7f88ed4f9640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f88e80d49f0 con 0x7f88e806c5c0 2024-04-25T10:59:40.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.611+0000 7f88e67fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88e8161710 0x7f88e80ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:40.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.611+0000 7f88e67fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88e8161710 0x7f88e80ce360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43692/0 (socket says 172.21.15.69:43692) 2024-04-25T10:59:40.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.611+0000 7f88e67fc640 1 -- 172.21.15.69:0/1233985235 learned_addr learned my addr 172.21.15.69:0/1233985235 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:40.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.611+0000 7f88e6ffd640 1 --2- 172.21.15.69:0/1233985235 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88e806c5c0 0x7f88e81523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:40.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.611+0000 7f88e77fe640 1 --2- 172.21.15.69:0/1233985235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88e8163fa0 0x7f88e80ba210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:40.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.611+0000 7f88e67fc640 1 -- 172.21.15.69:0/1233985235 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88e806c5c0 msgr2=0x7f88e81523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:40.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.611+0000 7f88e67fc640 1 --2- 172.21.15.69:0/1233985235 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88e806c5c0 0x7f88e81523f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:40.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.611+0000 7f88e67fc640 1 -- 172.21.15.69:0/1233985235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88e8163fa0 msgr2=0x7f88e80ba210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:40.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.611+0000 7f88e67fc640 1 --2- 172.21.15.69:0/1233985235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88e8163fa0 0x7f88e80ba210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:40.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.611+0000 7f88e67fc640 1 -- 172.21.15.69:0/1233985235 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e80cea10 con 0x7f88e8161710 2024-04-25T10:59:40.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.611+0000 7f88e6ffd640 1 --2- 172.21.15.69:0/1233985235 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88e806c5c0 0x7f88e81523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:59:40.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.611+0000 7f88e67fc640 1 --2- 172.21.15.69:0/1233985235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88e8161710 0x7f88e80ce360 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f88dc00c3f0 tx=0x7f88dc00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:40.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.611+0000 7f88cf7fe640 1 -- 172.21.15.69:0/1233985235 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc0074e0 con 0x7f88e8161710 2024-04-25T10:59:40.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.611+0000 7f88cf7fe640 1 -- 172.21.15.69:0/1233985235 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88dc007680 con 0x7f88e8161710 2024-04-25T10:59:40.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.611+0000 7f88ed4f9640 1 -- 172.21.15.69:0/1233985235 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88e80ced90 con 0x7f88e8161710 2024-04-25T10:59:40.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.611+0000 7f88cf7fe640 1 -- 172.21.15.69:0/1233985235 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc004a00 con 0x7f88e8161710 2024-04-25T10:59:40.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.611+0000 7f88ed4f9640 1 -- 172.21.15.69:0/1233985235 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88e80cf310 con 0x7f88e8161710 2024-04-25T10:59:40.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.613+0000 7f88ed4f9640 1 -- 172.21.15.69:0/1233985235 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88e8050740 con 0x7f88e8161710 2024-04-25T10:59:40.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.613+0000 7f88cf7fe640 1 -- 172.21.15.69:0/1233985235 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f88dc01a020 con 0x7f88e8161710 2024-04-25T10:59:40.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.613+0000 7f88cf7fe640 1 --2- 172.21.15.69:0/1233985235 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f88d40744b0 0x7f88d40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:40.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.614+0000 7f88cf7fe640 1 -- 172.21.15.69:0/1233985235 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f88dc0948a0 con 0x7f88e8161710 2024-04-25T10:59:40.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.614+0000 7f88e6ffd640 1 --2- 172.21.15.69:0/1233985235 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f88d40744b0 0x7f88d40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:40.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.614+0000 7f88e6ffd640 1 --2- 172.21.15.69:0/1233985235 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f88d40744b0 0x7f88d40769c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f88d8033000 tx=0x7f88d8009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:40.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.618+0000 7f88cf7fe640 1 -- 172.21.15.69:0/1233985235 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f88e80ced90 con 0x7f88e8161710 2024-04-25T10:59:40.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.793+0000 7f88ed4f9640 1 -- 172.21.15.69:0/1233985235 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f88e815eb10 con 0x7f88e8161710 2024-04-25T10:59:40.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.794+0000 7f88cf7fe640 1 -- 172.21.15.69:0/1233985235 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f88e81609b0 con 0x7f88e8161710 2024-04-25T10:59:40.794 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-25T10:59:40.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.796+0000 7f88ed4f9640 1 -- 172.21.15.69:0/1233985235 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f88d40744b0 msgr2=0x7f88d40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:40.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.796+0000 7f88ed4f9640 1 --2- 172.21.15.69:0/1233985235 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f88d40744b0 0x7f88d40769c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f88d8033000 tx=0x7f88d8009000 comp rx=0 tx=0).stop 2024-04-25T10:59:40.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.796+0000 7f88ed4f9640 1 -- 172.21.15.69:0/1233985235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88e8161710 msgr2=0x7f88e80ce360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:40.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.796+0000 7f88ed4f9640 1 --2- 172.21.15.69:0/1233985235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88e8161710 0x7f88e80ce360 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f88dc00c3f0 tx=0x7f88dc00c7f0 comp rx=0 tx=0).stop 2024-04-25T10:59:40.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.796+0000 7f88ed4f9640 1 -- 172.21.15.69:0/1233985235 shutdown_connections 2024-04-25T10:59:40.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.796+0000 7f88ed4f9640 1 --2- 172.21.15.69:0/1233985235 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f88d40744b0 0x7f88d40769c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:40.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.796+0000 7f88ed4f9640 1 --2- 172.21.15.69:0/1233985235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88e8163fa0 0x7f88e80ba210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:40.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.796+0000 7f88ed4f9640 1 --2- 172.21.15.69:0/1233985235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88e8161710 0x7f88e80ce360 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:40.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.796+0000 7f88ed4f9640 1 --2- 172.21.15.69:0/1233985235 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88e806c5c0 0x7f88e81523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:40.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.796+0000 7f88ed4f9640 1 -- 172.21.15.69:0/1233985235 >> 172.21.15.69:0/1233985235 conn(0x7f88e80d5d90 msgr2=0x7f88e815e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:40.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.796+0000 7f88ed4f9640 1 -- 172.21.15.69:0/1233985235 shutdown_connections 2024-04-25T10:59:40.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.796+0000 7f88ed4f9640 1 -- 172.21.15.69:0/1233985235 wait complete. 2024-04-25T10:59:40.807 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:59:40.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.914+0000 7f0dfead1640 1 Processor -- start 2024-04-25T10:59:40.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.914+0000 7f0dfead1640 1 -- start start 2024-04-25T10:59:40.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.915+0000 7f0dfead1640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0df80b8be0 0x7f0df80b9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:40.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.915+0000 7f0dfead1640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0df80b4ff0 0x7f0df80b6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:40.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.915+0000 7f0dfead1640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0df80d04b0 0x7f0df80d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:40.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.915+0000 7f0dfead1640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f0df80cee00 con 0x7f0df80b4ff0 2024-04-25T10:59:40.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.915+0000 7f0dfead1640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f0df80b5400 con 0x7f0df80d04b0 2024-04-25T10:59:40.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.915+0000 7f0dfead1640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0df80b6e50 con 0x7f0df80b8be0 2024-04-25T10:59:40.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.915+0000 7f0df7fff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0df80b4ff0 0x7f0df80b6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:40.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.915+0000 7f0df7fff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0df80b4ff0 0x7f0df80b6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43706/0 (socket says 172.21.15.69:43706) 2024-04-25T10:59:40.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.915+0000 7f0df7fff640 1 -- 172.21.15.69:0/2230825451 learned_addr learned my addr 172.21.15.69:0/2230825451 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:40.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.915+0000 7f0df7fff640 1 -- 172.21.15.69:0/2230825451 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0df80b8be0 msgr2=0x7f0df80b9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:40.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.915+0000 7f0dfc86d640 1 --2- 172.21.15.69:0/2230825451 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0df80b8be0 0x7f0df80b9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:40.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.915+0000 7f0dfd06e640 1 --2- 172.21.15.69:0/2230825451 >> v2:172.21.15.146:3300/0 conn(0x7f0df80d04b0 0x7f0df80d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:40.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.916+0000 7f0df7fff640 1 --2- 172.21.15.69:0/2230825451 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0df80b8be0 0x7f0df80b9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:40.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.916+0000 7f0df7fff640 1 -- 172.21.15.69:0/2230825451 >> v2:172.21.15.146:3300/0 conn(0x7f0df80d04b0 msgr2=0x7f0df80d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:40.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.916+0000 7f0df7fff640 1 --2- 172.21.15.69:0/2230825451 >> v2:172.21.15.146:3300/0 conn(0x7f0df80d04b0 0x7f0df80d08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:40.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.916+0000 7f0df7fff640 1 -- 172.21.15.69:0/2230825451 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df80d0e00 con 0x7f0df80b4ff0 2024-04-25T10:59:40.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.916+0000 7f0dfc86d640 1 --2- 172.21.15.69:0/2230825451 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0df80b8be0 0x7f0df80b9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:40.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.916+0000 7f0dfd06e640 1 --2- 172.21.15.69:0/2230825451 >> v2:172.21.15.146:3300/0 conn(0x7f0df80d04b0 0x7f0df80d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:40.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.916+0000 7f0df7fff640 1 --2- 172.21.15.69:0/2230825451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0df80b4ff0 0x7f0df80b6910 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0df8071240 tx=0x7f0de402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfae0d45f91d7a0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:40.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.916+0000 7f0df77fe640 1 -- 172.21.15.69:0/2230825451 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de4037070 con 0x7f0df80b4ff0 2024-04-25T10:59:40.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.916+0000 7f0df77fe640 1 -- 172.21.15.69:0/2230825451 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0de4032440 con 0x7f0df80b4ff0 2024-04-25T10:59:40.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.916+0000 7f0df77fe640 1 -- 172.21.15.69:0/2230825451 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de4032810 con 0x7f0df80b4ff0 2024-04-25T10:59:40.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.917+0000 7f0dfead1640 1 -- 172.21.15.69:0/2230825451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0df80b4ff0 msgr2=0x7f0df80b6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:40.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.917+0000 7f0dfead1640 1 --2- 172.21.15.69:0/2230825451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0df80b4ff0 0x7f0df80b6910 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0df8071240 tx=0x7f0de402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:59:40.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.917+0000 7f0dfead1640 1 -- 172.21.15.69:0/2230825451 shutdown_connections 2024-04-25T10:59:40.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.917+0000 7f0dfead1640 1 --2- 172.21.15.69:0/2230825451 >> v2:172.21.15.146:3300/0 conn(0x7f0df80d04b0 0x7f0df80d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:40.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.917+0000 7f0dfead1640 1 --2- 172.21.15.69:0/2230825451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0df80b4ff0 0x7f0df80b6910 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:40.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.917+0000 7f0dfead1640 1 --2- 172.21.15.69:0/2230825451 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0df80b8be0 0x7f0df80b9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:40.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.917+0000 7f0dfead1640 1 -- 172.21.15.69:0/2230825451 >> 172.21.15.69:0/2230825451 conn(0x7f0df80d3ac0 msgr2=0x7f0df80d3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:40.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.917+0000 7f0dfead1640 1 -- 172.21.15.69:0/2230825451 shutdown_connections 2024-04-25T10:59:40.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.917+0000 7f0dfead1640 1 -- 172.21.15.69:0/2230825451 wait complete. 2024-04-25T10:59:40.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.919+0000 7f0dfead1640 1 Processor -- start 2024-04-25T10:59:40.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.919+0000 7f0dfead1640 1 -- start start 2024-04-25T10:59:40.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.920+0000 7f0dfead1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0df80b4ff0 0x7f0df815bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:40.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.920+0000 7f0dfead1640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0df80b8be0 0x7f0df81f4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:40.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.920+0000 7f0dfead1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0df80d04b0 0x7f0df81f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:40.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.920+0000 7f0dfead1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0df80cea90 con 0x7f0df80b4ff0 2024-04-25T10:59:40.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.920+0000 7f0dfead1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0df80d2a00 con 0x7f0df80d04b0 2024-04-25T10:59:40.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.920+0000 7f0dfead1640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0df80d23d0 con 0x7f0df80b8be0 2024-04-25T10:59:40.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.921+0000 7f0dfc86d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0df80b4ff0 0x7f0df815bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:40.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.921+0000 7f0dfc86d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0df80b4ff0 0x7f0df815bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43712/0 (socket says 172.21.15.69:43712) 2024-04-25T10:59:40.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.921+0000 7f0dfc86d640 1 -- 172.21.15.69:0/279394882 learned_addr learned my addr 172.21.15.69:0/279394882 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:40.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.921+0000 7f0dfd06e640 1 --2- 172.21.15.69:0/279394882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0df80d04b0 0x7f0df81f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:40.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.921+0000 7f0df7fff640 1 --2- 172.21.15.69:0/279394882 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0df80b8be0 0x7f0df81f4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:40.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.921+0000 7f0dfc86d640 1 -- 172.21.15.69:0/279394882 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0df80b8be0 msgr2=0x7f0df81f4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:40.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.921+0000 7f0dfc86d640 1 --2- 172.21.15.69:0/279394882 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0df80b8be0 0x7f0df81f4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:40.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.921+0000 7f0dfc86d640 1 -- 172.21.15.69:0/279394882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0df80d04b0 msgr2=0x7f0df81f4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:40.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.921+0000 7f0dfc86d640 1 --2- 172.21.15.69:0/279394882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0df80d04b0 0x7f0df81f4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:40.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.921+0000 7f0dfc86d640 1 -- 172.21.15.69:0/279394882 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df80cc4e0 con 0x7f0df80b4ff0 2024-04-25T10:59:40.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.921+0000 7f0dfd06e640 1 --2- 172.21.15.69:0/279394882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0df80d04b0 0x7f0df81f4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:40.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.921+0000 7f0df7fff640 1 --2- 172.21.15.69:0/279394882 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0df80b8be0 0x7f0df81f4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:40.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.921+0000 7f0dfc86d640 1 --2- 172.21.15.69:0/279394882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0df80b4ff0 0x7f0df815bc20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0dec00c7f0 tx=0x7f0dec00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:40.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.921+0000 7f0df57fa640 1 -- 172.21.15.69:0/279394882 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dec004220 con 0x7f0df80b4ff0 2024-04-25T10:59:40.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.921+0000 7f0df57fa640 1 -- 172.21.15.69:0/279394882 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dec0043c0 con 0x7f0df80b4ff0 2024-04-25T10:59:40.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.921+0000 7f0dfead1640 1 -- 172.21.15.69:0/279394882 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0df80cb720 con 0x7f0df80b4ff0 2024-04-25T10:59:40.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.921+0000 7f0df57fa640 1 -- 172.21.15.69:0/279394882 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dec005bb0 con 0x7f0df80b4ff0 2024-04-25T10:59:40.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.921+0000 7f0dfead1640 1 -- 172.21.15.69:0/279394882 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0df80cbce0 con 0x7f0df80b4ff0 2024-04-25T10:59:40.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.922+0000 7f0dfead1640 1 -- 172.21.15.69:0/279394882 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0df8050740 con 0x7f0df80b4ff0 2024-04-25T10:59:40.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.923+0000 7f0df57fa640 1 -- 172.21.15.69:0/279394882 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0dec006030 con 0x7f0df80b4ff0 2024-04-25T10:59:40.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.923+0000 7f0df57fa640 1 --2- 172.21.15.69:0/279394882 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0ddc0744b0 0x7f0ddc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:40.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.924+0000 7f0df57fa640 1 -- 172.21.15.69:0/279394882 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0dec0957c0 con 0x7f0df80b4ff0 2024-04-25T10:59:40.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.924+0000 7f0df7fff640 1 --2- 172.21.15.69:0/279394882 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0ddc0744b0 0x7f0ddc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:40.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.924+0000 7f0df7fff640 1 --2- 172.21.15.69:0/279394882 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0ddc0744b0 0x7f0ddc0769c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0de4034000 tx=0x7f0de402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:40.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:40.928+0000 7f0df57fa640 1 -- 172.21.15.69:0/279394882 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0dec01e020 con 0x7f0df80b4ff0 2024-04-25T10:59:41.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.106+0000 7f0dfead1640 1 -- 172.21.15.69:0/279394882 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0df80687c0 con 0x7f0df80b4ff0 2024-04-25T10:59:41.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.107+0000 7f0df57fa640 1 -- 172.21.15.69:0/279394882 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f0dec015030 con 0x7f0df80b4ff0 2024-04-25T10:59:41.108 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:41.108 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":20,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T10:59:20.630151+0000","last_up_change":"2024-04-25T10:59:05.546839+0000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:59:06.422674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:59:16.727980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:59:17.938486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6825","nonce":1876101065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6827","nonce":1876101065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6831","nonce":1876101065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6829","nonce":1876101065}]},"public_addr":"172.21.15.69:6825/1876101065","cluster_addr":"172.21.15.69:6827/1876101065","heartbeat_back_addr":"172.21.15.69:6831/1876101065","heartbeat_front_addr":"172.21.15.69:6829/1876101065","state":["exists","up"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6801","nonce":1806438563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6803","nonce":1806438563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6807","nonce":1806438563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6805","nonce":1806438563}]},"public_addr":"172.21.15.69:6801/1806438563","cluster_addr":"172.21.15.69:6803/1806438563","heartbeat_back_addr":"172.21.15.69:6807/1806438563","heartbeat_front_addr":"172.21.15.69:6805/1806438563","state":["exists","up"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6817","nonce":1593617983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6819","nonce":1593617983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6823","nonce":1593617983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6821","nonce":1593617983}]},"public_addr":"172.21.15.69:6817/1593617983","cluster_addr":"172.21.15.69:6819/1593617983","heartbeat_back_addr":"172.21.15.69:6823/1593617983","heartbeat_front_addr":"172.21.15.69:6821/1593617983","state":["exists","up"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6809","nonce":3950661294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6811","nonce":3950661294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6815","nonce":3950661294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6813","nonce":3950661294}]},"public_addr":"172.21.15.69:6809/3950661294","cluster_addr":"172.21.15.69:6811/3950661294","heartbeat_back_addr":"172.21.15.69:6815/3950661294","heartbeat_front_addr":"172.21.15.69:6813/3950661294","state":["exists","up"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6801","nonce":2508929872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6803","nonce":2508929872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6807","nonce":2508929872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6805","nonce":2508929872}]},"public_addr":"172.21.15.146:6801/2508929872","cluster_addr":"172.21.15.146:6803/2508929872","heartbeat_back_addr":"172.21.15.146:6807/2508929872","heartbeat_front_addr":"172.21.15.146:6805/2508929872","state":["exists","up"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6811","nonce":1310911138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6816","nonce":1310911138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6823","nonce":1310911138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6821","nonce":1310911138}]},"public_addr":"172.21.15.146:6811/1310911138","cluster_addr":"172.21.15.146:6816/1310911138","heartbeat_back_addr":"172.21.15.146:6823/1310911138","heartbeat_front_addr":"172.21.15.146:6821/1310911138","state":["exists","up"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6810","nonce":634688021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6813","nonce":634688021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6820","nonce":634688021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6817","nonce":634688021}]},"public_addr":"172.21.15.146:6810/634688021","cluster_addr":"172.21.15.146:6813/634688021","heartbeat_back_addr":"172.21.15.146:6820/634688021","heartbeat_front_addr":"172.21.15.146:6817/634688021","state":["exists","up"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6825","nonce":3814215935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6827","nonce":3814215935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6831","nonce":3814215935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6829","nonce":3814215935}]},"public_addr":"172.21.15.146:6825/3814215935","cluster_addr":"172.21.15.146:6827/3814215935","heartbeat_back_addr":"172.21.15.146:6831/3814215935","heartbeat_front_addr":"172.21.15.146:6829/3814215935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.820200+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.850189+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.764560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.866248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.807897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.869881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.782310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.827131+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:59:41.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.109+0000 7f0dfead1640 1 -- 172.21.15.69:0/279394882 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0ddc0744b0 msgr2=0x7f0ddc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:41.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.109+0000 7f0dfead1640 1 --2- 172.21.15.69:0/279394882 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0ddc0744b0 0x7f0ddc0769c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0de4034000 tx=0x7f0de402e000 comp rx=0 tx=0).stop 2024-04-25T10:59:41.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.109+0000 7f0dfead1640 1 -- 172.21.15.69:0/279394882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0df80b4ff0 msgr2=0x7f0df815bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:41.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.109+0000 7f0dfead1640 1 --2- 172.21.15.69:0/279394882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0df80b4ff0 0x7f0df815bc20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0dec00c7f0 tx=0x7f0dec00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:59:41.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.109+0000 7f0dfead1640 1 -- 172.21.15.69:0/279394882 shutdown_connections 2024-04-25T10:59:41.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.109+0000 7f0dfead1640 1 --2- 172.21.15.69:0/279394882 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0ddc0744b0 0x7f0ddc0769c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:41.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.110+0000 7f0dfead1640 1 --2- 172.21.15.69:0/279394882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0df80d04b0 0x7f0df81f4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:41.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.110+0000 7f0dfead1640 1 --2- 172.21.15.69:0/279394882 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0df80b8be0 0x7f0df81f4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:41.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.110+0000 7f0dfead1640 1 --2- 172.21.15.69:0/279394882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0df80b4ff0 0x7f0df815bc20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:41.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.110+0000 7f0dfead1640 1 -- 172.21.15.69:0/279394882 >> 172.21.15.69:0/279394882 conn(0x7f0df80d3ac0 msgr2=0x7f0df80715a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:41.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.110+0000 7f0dfead1640 1 -- 172.21.15.69:0/279394882 shutdown_connections 2024-04-25T10:59:41.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.110+0000 7f0dfead1640 1 -- 172.21.15.69:0/279394882 wait complete. 2024-04-25T10:59:41.122 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:59:06.422674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:59:16.727980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:59:17.938486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:59:41.123 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:59:41.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.231+0000 7f13fbd46640 1 Processor -- start 2024-04-25T10:59:41.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.231+0000 7f13fbd46640 1 -- start start 2024-04-25T10:59:41.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.233+0000 7f13fbd46640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f13f406dce0 0x7f13f40704c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:41.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.233+0000 7f13fbd46640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13f415adf0 0x7f13f40cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:41.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.233+0000 7f13fbd46640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13f406fa20 con 0x7f13f40b0c90 2024-04-25T10:59:41.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.233+0000 7f13fbd46640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f13f4070fd0 con 0x7f13f406dce0 2024-04-25T10:59:41.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.233+0000 7f13fbd46640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f13f40712b0 con 0x7f13f415adf0 2024-04-25T10:59:41.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.233+0000 7f13f9ae2640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f13f40b0c90 0x7f13f406d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:44290/0 (socket says 172.21.15.69:44290) 2024-04-25T10:59:41.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.233+0000 7f13f9ae2640 1 -- 172.21.15.69:0/1448029781 learned_addr learned my addr 172.21.15.69:0/1448029781 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:41.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.233+0000 7f13f8ae0640 1 -- 172.21.15.69:0/1448029781 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3717247079 0 0) 0x7f13f406fa20 con 0x7f13f40b0c90 2024-04-25T10:59:41.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.233+0000 7f13f92e1640 1 --2- 172.21.15.69:0/1448029781 >> v2:172.21.15.146:3300/0 conn(0x7f13f406dce0 0x7f13f40704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:41.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.233+0000 7f13f8ae0640 1 -- 172.21.15.69:0/1448029781 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13e0003520 con 0x7f13f40b0c90 2024-04-25T10:59:41.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.233+0000 7f13fa2e3640 1 --2- 172.21.15.69:0/1448029781 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13f415adf0 0x7f13f40cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:41.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.233+0000 7f13f8ae0640 1 -- 172.21.15.69:0/1448029781 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 766753929 0 0) 0x7f13e0003520 con 0x7f13f40b0c90 2024-04-25T10:59:41.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.233+0000 7f13f8ae0640 1 -- 172.21.15.69:0/1448029781 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13f415adf0 msgr2=0x7f13f40cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:41.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.233+0000 7f13f8ae0640 1 --2- 172.21.15.69:0/1448029781 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13f415adf0 0x7f13f40cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:41.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.233+0000 7f13f8ae0640 1 -- 172.21.15.69:0/1448029781 >> v2:172.21.15.146:3300/0 conn(0x7f13f406dce0 msgr2=0x7f13f40704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:41.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.233+0000 7f13f8ae0640 1 --2- 172.21.15.69:0/1448029781 >> v2:172.21.15.146:3300/0 conn(0x7f13f406dce0 0x7f13f40704c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:41.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.233+0000 7f13f8ae0640 1 -- 172.21.15.69:0/1448029781 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f406fcc0 con 0x7f13f40b0c90 2024-04-25T10:59:41.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.233+0000 7f13f8ae0640 1 -- 172.21.15.69:0/1448029781 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7f13f0001670 con 0x7f13f40b0c90 2024-04-25T10:59:41.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.233+0000 7f13f8ae0640 1 -- 172.21.15.69:0/1448029781 >> v1:172.21.15.69:6789/0 conn(0x7f13f40b0c90 legacy=0x7f13f406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:41.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.234+0000 7f13f8ae0640 1 --2- 172.21.15.69:0/1448029781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13e00039b0 0x7f13e0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:41.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.234+0000 7f13f8ae0640 1 --2- 172.21.15.69:0/1448029781 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13f415adf0 0x7f13e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:41.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.234+0000 7f13f8ae0640 1 --2- 172.21.15.69:0/1448029781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13e0004d40 0x7f13e0005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:41.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.234+0000 7f13f8ae0640 1 -- 172.21.15.69:0/1448029781 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f13f4070fd0 con 0x7f13e00039b0 2024-04-25T10:59:41.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.234+0000 7f13f8ae0640 1 -- 172.21.15.69:0/1448029781 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f13f40712b0 con 0x7f13e0004d40 2024-04-25T10:59:41.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.234+0000 7f13f8ae0640 1 -- 172.21.15.69:0/1448029781 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f13e000b9f0 con 0x7f13f415adf0 2024-04-25T10:59:41.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.234+0000 7f13f9ae2640 1 --2- 172.21.15.69:0/1448029781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13e00039b0 0x7f13e0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:41.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.234+0000 7f13f92e1640 1 --2- 172.21.15.69:0/1448029781 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13f415adf0 0x7f13e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:41.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.234+0000 7f13fa2e3640 1 --2- 172.21.15.69:0/1448029781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13e0004d40 0x7f13e0005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:41.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.234+0000 7f13f9ae2640 1 -- 172.21.15.69:0/1448029781 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13f415adf0 msgr2=0x7f13e0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:41.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.234+0000 7f13f9ae2640 1 --2- 172.21.15.69:0/1448029781 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13f415adf0 0x7f13e0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:41.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.234+0000 7f13f9ae2640 1 -- 172.21.15.69:0/1448029781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13e0004d40 msgr2=0x7f13e0005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:41.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.234+0000 7f13f9ae2640 1 --2- 172.21.15.69:0/1448029781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13e0004d40 0x7f13e0005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:41.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.234+0000 7f13f9ae2640 1 -- 172.21.15.69:0/1448029781 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e0003520 con 0x7f13e00039b0 2024-04-25T10:59:41.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.234+0000 7f13fa2e3640 1 --2- 172.21.15.69:0/1448029781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13e0004d40 0x7f13e0005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:41.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.235+0000 7f13f9ae2640 1 --2- 172.21.15.69:0/1448029781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13e00039b0 0x7f13e0003e20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f13f0001eb0 tx=0x7f13e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=521773c2bb059a2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:41.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.235+0000 7f13f8ae0640 1 -- 172.21.15.69:0/1448029781 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f0007d70 con 0x7f13e00039b0 2024-04-25T10:59:41.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.235+0000 7f13f8ae0640 1 -- 172.21.15.69:0/1448029781 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13f0005af0 con 0x7f13e00039b0 2024-04-25T10:59:41.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.235+0000 7f13f8ae0640 1 -- 172.21.15.69:0/1448029781 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f0007070 con 0x7f13e00039b0 2024-04-25T10:59:41.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.235+0000 7f13fbd46640 1 -- 172.21.15.69:0/1448029781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13e00039b0 msgr2=0x7f13e0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:41.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.235+0000 7f13fbd46640 1 --2- 172.21.15.69:0/1448029781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13e00039b0 0x7f13e0003e20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f13f0001eb0 tx=0x7f13e0005830 comp rx=0 tx=0).stop 2024-04-25T10:59:41.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.235+0000 7f13fa2e3640 1 -- 172.21.15.69:0/1448029781 reap_dead start 2024-04-25T10:59:41.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.235+0000 7f13fbd46640 1 -- 172.21.15.69:0/1448029781 shutdown_connections 2024-04-25T10:59:41.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.235+0000 7f13fbd46640 1 -- 172.21.15.69:0/1448029781 >> 172.21.15.69:0/1448029781 conn(0x7f13f4073090 msgr2=0x7f13f40734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:41.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.236+0000 7f13fbd46640 1 -- 172.21.15.69:0/1448029781 shutdown_connections 2024-04-25T10:59:41.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.236+0000 7f13fbd46640 1 -- 172.21.15.69:0/1448029781 wait complete. 2024-04-25T10:59:41.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.237+0000 7f13fbd46640 1 Processor -- start 2024-04-25T10:59:41.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.237+0000 7f13fbd46640 1 -- start start 2024-04-25T10:59:41.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.238+0000 7f13fbd46640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13e00039b0 0x7f13f4153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:41.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.238+0000 7f13fbd46640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13f41472e0 0x7f13f40aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:41.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.238+0000 7f13fbd46640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13f40af430 0x7f13f4158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:41.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.239+0000 7f13fbd46640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f13f40cec50 con 0x7f13f40af430 2024-04-25T10:59:41.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.239+0000 7f13fbd46640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f13f4070c80 con 0x7f13f41472e0 2024-04-25T10:59:41.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.239+0000 7f13fbd46640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f13f40cedd0 con 0x7f13e00039b0 2024-04-25T10:59:41.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.239+0000 7f13fa2e3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13f40af430 0x7f13f4158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:41.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.239+0000 7f13fa2e3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13f40af430 0x7f13f4158470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43726/0 (socket says 172.21.15.69:43726) 2024-04-25T10:59:41.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.239+0000 7f13fa2e3640 1 -- 172.21.15.69:0/3001407365 learned_addr learned my addr 172.21.15.69:0/3001407365 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:41.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.239+0000 7f13f9ae2640 1 --2- 172.21.15.69:0/3001407365 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13e00039b0 0x7f13f4153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:41.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.239+0000 7f13f92e1640 1 --2- 172.21.15.69:0/3001407365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13f41472e0 0x7f13f40aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:41.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.239+0000 7f13fa2e3640 1 -- 172.21.15.69:0/3001407365 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13e00039b0 msgr2=0x7f13f4153d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:41.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.239+0000 7f13fa2e3640 1 --2- 172.21.15.69:0/3001407365 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13e00039b0 0x7f13f4153d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:41.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.239+0000 7f13fa2e3640 1 -- 172.21.15.69:0/3001407365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13f41472e0 msgr2=0x7f13f40aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:41.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.239+0000 7f13fa2e3640 1 --2- 172.21.15.69:0/3001407365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13f41472e0 0x7f13f40aeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:41.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.239+0000 7f13fa2e3640 1 -- 172.21.15.69:0/3001407365 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f4152f90 con 0x7f13f40af430 2024-04-25T10:59:41.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.239+0000 7f13fa2e3640 1 --2- 172.21.15.69:0/3001407365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13f40af430 0x7f13f4158470 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f13f406fbf0 tx=0x7f13ec010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:41.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.239+0000 7f13ea7fc640 1 -- 172.21.15.69:0/3001407365 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ec010d80 con 0x7f13f40af430 2024-04-25T10:59:41.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.239+0000 7f13ea7fc640 1 -- 172.21.15.69:0/3001407365 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13ec003990 con 0x7f13f40af430 2024-04-25T10:59:41.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.239+0000 7f13fbd46640 1 -- 172.21.15.69:0/3001407365 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13f4153270 con 0x7f13f40af430 2024-04-25T10:59:41.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.239+0000 7f13ea7fc640 1 -- 172.21.15.69:0/3001407365 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ec003d80 con 0x7f13f40af430 2024-04-25T10:59:41.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.239+0000 7f13fbd46640 1 -- 172.21.15.69:0/3001407365 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13f4153830 con 0x7f13f40af430 2024-04-25T10:59:41.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.240+0000 7f13fbd46640 1 -- 172.21.15.69:0/3001407365 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13f4050740 con 0x7f13f40af430 2024-04-25T10:59:41.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.241+0000 7f13ea7fc640 1 -- 172.21.15.69:0/3001407365 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f13ec005d10 con 0x7f13f40af430 2024-04-25T10:59:41.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.241+0000 7f13ea7fc640 1 --2- 172.21.15.69:0/3001407365 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f13d80744b0 0x7f13d80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:41.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.241+0000 7f13ea7fc640 1 -- 172.21.15.69:0/3001407365 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f13ec095780 con 0x7f13f40af430 2024-04-25T10:59:41.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.241+0000 7f13f9ae2640 1 --2- 172.21.15.69:0/3001407365 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f13d80744b0 0x7f13d80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:41.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.242+0000 7f13f9ae2640 1 --2- 172.21.15.69:0/3001407365 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f13d80744b0 0x7f13d80769c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f13f0033000 tx=0x7f13f0035040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:41.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.246+0000 7f13ea7fc640 1 -- 172.21.15.69:0/3001407365 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f13ec025020 con 0x7f13f40af430 2024-04-25T10:59:41.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.419+0000 7f13fbd46640 1 -- 172.21.15.69:0/3001407365 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f13f40b0070 con 0x7f13f40af430 2024-04-25T10:59:41.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.419+0000 7f13ea7fc640 1 -- 172.21.15.69:0/3001407365 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f13ec05f950 con 0x7f13f40af430 2024-04-25T10:59:41.421 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-25T10:59:41.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.421+0000 7f13fbd46640 1 -- 172.21.15.69:0/3001407365 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f13d80744b0 msgr2=0x7f13d80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:41.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.421+0000 7f13fbd46640 1 --2- 172.21.15.69:0/3001407365 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f13d80744b0 0x7f13d80769c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f13f0033000 tx=0x7f13f0035040 comp rx=0 tx=0).stop 2024-04-25T10:59:41.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.421+0000 7f13fbd46640 1 -- 172.21.15.69:0/3001407365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13f40af430 msgr2=0x7f13f4158470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:41.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.421+0000 7f13fbd46640 1 --2- 172.21.15.69:0/3001407365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13f40af430 0x7f13f4158470 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f13f406fbf0 tx=0x7f13ec010590 comp rx=0 tx=0).stop 2024-04-25T10:59:41.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.421+0000 7f13fbd46640 1 -- 172.21.15.69:0/3001407365 shutdown_connections 2024-04-25T10:59:41.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.421+0000 7f13fbd46640 1 --2- 172.21.15.69:0/3001407365 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f13d80744b0 0x7f13d80769c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:41.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.421+0000 7f13fbd46640 1 --2- 172.21.15.69:0/3001407365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13f40af430 0x7f13f4158470 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:41.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.421+0000 7f13fbd46640 1 --2- 172.21.15.69:0/3001407365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13f41472e0 0x7f13f40aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:41.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.421+0000 7f13fbd46640 1 --2- 172.21.15.69:0/3001407365 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f13e00039b0 0x7f13f4153d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:41.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.421+0000 7f13fbd46640 1 -- 172.21.15.69:0/3001407365 >> 172.21.15.69:0/3001407365 conn(0x7f13f4073090 msgr2=0x7f13f406ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:41.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.422+0000 7f13fbd46640 1 -- 172.21.15.69:0/3001407365 shutdown_connections 2024-04-25T10:59:41.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.422+0000 7f13fbd46640 1 -- 172.21.15.69:0/3001407365 wait complete. 2024-04-25T10:59:41.434 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:59:41.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.543+0000 7f3325ad1640 1 Processor -- start 2024-04-25T10:59:41.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.543+0000 7f3325ad1640 1 -- start start 2024-04-25T10:59:41.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.545+0000 7f3325ad1640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33200b2e60 0x7f33200d3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:41.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.545+0000 7f3325ad1640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33200d23f0 con 0x7f3320164020 2024-04-25T10:59:41.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.545+0000 7f3325ad1640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33200d26d0 con 0x7f33200d1e20 2024-04-25T10:59:41.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.545+0000 7f3325ad1640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f33200d29b0 con 0x7f33200b2e60 2024-04-25T10:59:41.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.545+0000 7f331ffff640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f3320164020 0x7f3320164430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:44292/0 (socket says 172.21.15.69:44292) 2024-04-25T10:59:41.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.545+0000 7f331ffff640 1 -- 172.21.15.69:0/3128142062 learned_addr learned my addr 172.21.15.69:0/3128142062 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:41.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.545+0000 7f331e7fc640 1 -- 172.21.15.69:0/3128142062 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3653745671 0 0) 0x7f33200d23f0 con 0x7f3320164020 2024-04-25T10:59:41.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.545+0000 7f331effd640 1 --2- 172.21.15.69:0/3128142062 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33200b2e60 0x7f33200d3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:41.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.545+0000 7f331e7fc640 1 -- 172.21.15.69:0/3128142062 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3308003520 con 0x7f3320164020 2024-04-25T10:59:41.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.545+0000 7f331e7fc640 1 -- 172.21.15.69:0/3128142062 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1018239991 0 0) 0x7f3308003520 con 0x7f3320164020 2024-04-25T10:59:41.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.545+0000 7f331e7fc640 1 -- 172.21.15.69:0/3128142062 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33200b2e60 msgr2=0x7f33200d3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:41.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.545+0000 7f331e7fc640 1 --2- 172.21.15.69:0/3128142062 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33200b2e60 0x7f33200d3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:41.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.545+0000 7f331e7fc640 1 -- 172.21.15.69:0/3128142062 >> v1:172.21.15.146:6789/0 conn(0x7f33200d1e20 legacy=0x7f332008d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:41.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.545+0000 7f331e7fc640 1 -- 172.21.15.69:0/3128142062 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33200d2c90 con 0x7f3320164020 2024-04-25T10:59:41.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.545+0000 7f331e7fc640 1 -- 172.21.15.69:0/3128142062 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7f33180019f0 con 0x7f3320164020 2024-04-25T10:59:41.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.545+0000 7f331e7fc640 1 -- 172.21.15.69:0/3128142062 >> v1:172.21.15.69:6789/0 conn(0x7f3320164020 legacy=0x7f3320164430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:41.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.545+0000 7f331e7fc640 1 --2- 172.21.15.69:0/3128142062 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33200b2e60 0x7f33080039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:41.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.545+0000 7f331e7fc640 1 --2- 172.21.15.69:0/3128142062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3308003f60 0x7f3308004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:41.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.545+0000 7f331e7fc640 1 --2- 172.21.15.69:0/3128142062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3308004d40 0x7f3308005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:41.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.545+0000 7f331e7fc640 1 -- 172.21.15.69:0/3128142062 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f33200d29b0 con 0x7f3308003f60 2024-04-25T10:59:41.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.545+0000 7f331e7fc640 1 -- 172.21.15.69:0/3128142062 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f330800bbe0 con 0x7f3308004d40 2024-04-25T10:59:41.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.545+0000 7f331e7fc640 1 -- 172.21.15.69:0/3128142062 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f330800bde0 con 0x7f33200b2e60 2024-04-25T10:59:41.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.546+0000 7f331effd640 1 --2- 172.21.15.69:0/3128142062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3308003f60 0x7f3308004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:41.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.546+0000 7f331f7fe640 1 --2- 172.21.15.69:0/3128142062 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33200b2e60 0x7f33080039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:41.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.546+0000 7f331ffff640 1 --2- 172.21.15.69:0/3128142062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3308004d40 0x7f3308005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:41.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.546+0000 7f331effd640 1 -- 172.21.15.69:0/3128142062 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33200b2e60 msgr2=0x7f33080039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:41.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.546+0000 7f331effd640 1 --2- 172.21.15.69:0/3128142062 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33200b2e60 0x7f33080039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:41.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.546+0000 7f331effd640 1 -- 172.21.15.69:0/3128142062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3308004d40 msgr2=0x7f3308005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:41.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.546+0000 7f331effd640 1 --2- 172.21.15.69:0/3128142062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3308004d40 0x7f3308005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:41.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.546+0000 7f331effd640 1 -- 172.21.15.69:0/3128142062 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3308003520 con 0x7f3308003f60 2024-04-25T10:59:41.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.546+0000 7f331f7fe640 1 --2- 172.21.15.69:0/3128142062 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33200b2e60 0x7f33080039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:41.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.546+0000 7f331effd640 1 --2- 172.21.15.69:0/3128142062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3308003f60 0x7f3308004800 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3314009ac0 tx=0x7f3308005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d2c24e525c6a62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:41.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.547+0000 7f331e7fc640 1 -- 172.21.15.69:0/3128142062 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314036070 con 0x7f3308003f60 2024-04-25T10:59:41.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.547+0000 7f331e7fc640 1 -- 172.21.15.69:0/3128142062 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3314031400 con 0x7f3308003f60 2024-04-25T10:59:41.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.547+0000 7f331e7fc640 1 -- 172.21.15.69:0/3128142062 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314009690 con 0x7f3308003f60 2024-04-25T10:59:41.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.547+0000 7f3325ad1640 1 -- 172.21.15.69:0/3128142062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3308003f60 msgr2=0x7f3308004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:41.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.547+0000 7f3325ad1640 1 --2- 172.21.15.69:0/3128142062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3308003f60 0x7f3308004800 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3314009ac0 tx=0x7f3308005830 comp rx=0 tx=0).stop 2024-04-25T10:59:41.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.547+0000 7f331ffff640 1 -- 172.21.15.69:0/3128142062 reap_dead start 2024-04-25T10:59:41.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.547+0000 7f3325ad1640 1 -- 172.21.15.69:0/3128142062 shutdown_connections 2024-04-25T10:59:41.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.547+0000 7f3325ad1640 1 -- 172.21.15.69:0/3128142062 >> 172.21.15.69:0/3128142062 conn(0x7f33200d0530 msgr2=0x7f33200d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:41.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.547+0000 7f3325ad1640 1 -- 172.21.15.69:0/3128142062 shutdown_connections 2024-04-25T10:59:41.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.547+0000 7f3325ad1640 1 -- 172.21.15.69:0/3128142062 wait complete. 2024-04-25T10:59:41.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.549+0000 7f3325ad1640 1 Processor -- start 2024-04-25T10:59:41.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.549+0000 7f3325ad1640 1 -- start start 2024-04-25T10:59:41.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.550+0000 7f3325ad1640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3308003f60 0x7f332015d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:41.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.550+0000 7f3325ad1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f332014aea0 0x7f3320072620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:41.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.550+0000 7f3325ad1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3320072b60 0x7f33200b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:41.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.550+0000 7f3325ad1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f33200d3610 con 0x7f332014aea0 2024-04-25T10:59:41.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.550+0000 7f3325ad1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f33200d3490 con 0x7f3320072b60 2024-04-25T10:59:41.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.550+0000 7f3325ad1640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f33200d3790 con 0x7f3308003f60 2024-04-25T10:59:41.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.551+0000 7f331effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f332014aea0 0x7f3320072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:41.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.551+0000 7f331effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f332014aea0 0x7f3320072620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43748/0 (socket says 172.21.15.69:43748) 2024-04-25T10:59:41.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.551+0000 7f331effd640 1 -- 172.21.15.69:0/3306395654 learned_addr learned my addr 172.21.15.69:0/3306395654 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:41.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.551+0000 7f331ffff640 1 --2- 172.21.15.69:0/3306395654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3320072b60 0x7f33200b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:41.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.551+0000 7f331f7fe640 1 --2- 172.21.15.69:0/3306395654 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3308003f60 0x7f332015d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:41.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.551+0000 7f331effd640 1 -- 172.21.15.69:0/3306395654 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3308003f60 msgr2=0x7f332015d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:41.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.551+0000 7f331effd640 1 --2- 172.21.15.69:0/3306395654 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3308003f60 0x7f332015d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:41.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.551+0000 7f331effd640 1 -- 172.21.15.69:0/3306395654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3320072b60 msgr2=0x7f33200b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:41.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.551+0000 7f331effd640 1 --2- 172.21.15.69:0/3306395654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3320072b60 0x7f33200b3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:41.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.551+0000 7f331effd640 1 -- 172.21.15.69:0/3306395654 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3320071580 con 0x7f332014aea0 2024-04-25T10:59:41.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.551+0000 7f331effd640 1 --2- 172.21.15.69:0/3306395654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f332014aea0 0x7f3320072620 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3314009be0 tx=0x7f3314004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:41.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.551+0000 7f3307fff640 1 -- 172.21.15.69:0/3306395654 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314040020 con 0x7f332014aea0 2024-04-25T10:59:41.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.551+0000 7f3307fff640 1 -- 172.21.15.69:0/3306395654 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3314003c40 con 0x7f332014aea0 2024-04-25T10:59:41.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.551+0000 7f3325ad1640 1 -- 172.21.15.69:0/3306395654 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3320071860 con 0x7f332014aea0 2024-04-25T10:59:41.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.551+0000 7f3307fff640 1 -- 172.21.15.69:0/3306395654 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314036040 con 0x7f332014aea0 2024-04-25T10:59:41.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.551+0000 7f3325ad1640 1 -- 172.21.15.69:0/3306395654 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3320071e20 con 0x7f332014aea0 2024-04-25T10:59:41.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.552+0000 7f3325ad1640 1 -- 172.21.15.69:0/3306395654 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3320050740 con 0x7f332014aea0 2024-04-25T10:59:41.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.553+0000 7f3307fff640 1 -- 172.21.15.69:0/3306395654 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3314043020 con 0x7f332014aea0 2024-04-25T10:59:41.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.553+0000 7f3307fff640 1 --2- 172.21.15.69:0/3306395654 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f330c0744b0 0x7f330c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:41.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.553+0000 7f331f7fe640 1 --2- 172.21.15.69:0/3306395654 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f330c0744b0 0x7f330c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:41.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.553+0000 7f3307fff640 1 -- 172.21.15.69:0/3306395654 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f33140b5920 con 0x7f332014aea0 2024-04-25T10:59:41.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.554+0000 7f331f7fe640 1 --2- 172.21.15.69:0/3306395654 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f330c0744b0 0x7f330c0769c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3310005550 tx=0x7f3310009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:41.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.558+0000 7f3307fff640 1 -- 172.21.15.69:0/3306395654 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f331404a030 con 0x7f332014aea0 2024-04-25T10:59:41.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.735+0000 7f3325ad1640 1 -- 172.21.15.69:0/3306395654 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3320071000 con 0x7f332014aea0 2024-04-25T10:59:41.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.735+0000 7f3307fff640 1 -- 172.21.15.69:0/3306395654 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f33140800b0 con 0x7f332014aea0 2024-04-25T10:59:41.736 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-25T10:59:41.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.737+0000 7f3325ad1640 1 -- 172.21.15.69:0/3306395654 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f330c0744b0 msgr2=0x7f330c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:41.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.737+0000 7f3325ad1640 1 --2- 172.21.15.69:0/3306395654 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f330c0744b0 0x7f330c0769c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3310005550 tx=0x7f3310009040 comp rx=0 tx=0).stop 2024-04-25T10:59:41.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.737+0000 7f3325ad1640 1 -- 172.21.15.69:0/3306395654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f332014aea0 msgr2=0x7f3320072620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:41.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.737+0000 7f3325ad1640 1 --2- 172.21.15.69:0/3306395654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f332014aea0 0x7f3320072620 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3314009be0 tx=0x7f3314004730 comp rx=0 tx=0).stop 2024-04-25T10:59:41.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.737+0000 7f3325ad1640 1 -- 172.21.15.69:0/3306395654 shutdown_connections 2024-04-25T10:59:41.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.737+0000 7f3325ad1640 1 --2- 172.21.15.69:0/3306395654 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f330c0744b0 0x7f330c0769c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:41.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.737+0000 7f3325ad1640 1 --2- 172.21.15.69:0/3306395654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3320072b60 0x7f33200b3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:41.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.737+0000 7f3325ad1640 1 --2- 172.21.15.69:0/3306395654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f332014aea0 0x7f3320072620 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:41.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.737+0000 7f3325ad1640 1 --2- 172.21.15.69:0/3306395654 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3308003f60 0x7f332015d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:41.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.737+0000 7f3325ad1640 1 -- 172.21.15.69:0/3306395654 >> 172.21.15.69:0/3306395654 conn(0x7f33200d0530 msgr2=0x7f33200b0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:41.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.737+0000 7f3325ad1640 1 -- 172.21.15.69:0/3306395654 shutdown_connections 2024-04-25T10:59:41.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.737+0000 7f3325ad1640 1 -- 172.21.15.69:0/3306395654 wait complete. 2024-04-25T10:59:41.749 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:59:41.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.858+0000 7f4b432af640 1 Processor -- start 2024-04-25T10:59:41.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.858+0000 7f4b432af640 1 -- start start 2024-04-25T10:59:41.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.859+0000 7f4b432af640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b3c0cec90 0x7f4b3c0d2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:41.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.859+0000 7f4b432af640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f4b3c0b3f20 0x7f4b3c0b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:41.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.859+0000 7f4b432af640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b3c0ce5d0 con 0x7f4b3c0ce1c0 2024-04-25T10:59:41.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.859+0000 7f4b432af640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f4b3c0cf3c0 con 0x7f4b3c0b3f20 2024-04-25T10:59:41.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.859+0000 7f4b432af640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4b3c0cf6a0 con 0x7f4b3c0cec90 2024-04-25T10:59:41.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.859+0000 7f4b4184c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f4b3c0b3f20 0x7f4b3c0b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:41.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.859+0000 7f4b4104b640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f4b3c0ce1c0 0x7f4b3c0d2580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:44298/0 (socket says 172.21.15.69:44298) 2024-04-25T10:59:41.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.859+0000 7f4b4104b640 1 -- 172.21.15.69:0/469834052 learned_addr learned my addr 172.21.15.69:0/469834052 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:41.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.859+0000 7f4b4084a640 1 --2- 172.21.15.69:0/469834052 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b3c0cec90 0x7f4b3c0d2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:41.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.860+0000 7f4b4184c640 1 -- 172.21.15.69:0/469834052 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b3c0cec90 msgr2=0x7f4b3c0d2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:41.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.860+0000 7f4b4184c640 1 --2- 172.21.15.69:0/469834052 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b3c0cec90 0x7f4b3c0d2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:41.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.860+0000 7f4b4184c640 1 -- 172.21.15.69:0/469834052 >> v1:172.21.15.69:6789/0 conn(0x7f4b3c0ce1c0 legacy=0x7f4b3c0d2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:41.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.860+0000 7f4b4184c640 1 -- 172.21.15.69:0/469834052 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b3c0cf980 con 0x7f4b3c0b3f20 2024-04-25T10:59:41.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.860+0000 7f4b33fff640 1 -- 172.21.15.69:0/469834052 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 998036127 0 0) 0x7f4b3c0ce5d0 con 0x7f4b3c0ce1c0 2024-04-25T10:59:41.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.860+0000 7f4b4084a640 1 --2- 172.21.15.69:0/469834052 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b3c0cec90 0x7f4b3c0d2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:41.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.860+0000 7f4b4184c640 1 --2- 172.21.15.69:0/469834052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b3c0b3f20 0x7f4b3c0b4330 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009d90 tx=0x7f4b3402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4fc2be09b8544c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:41.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.860+0000 7f4b33fff640 1 -- 172.21.15.69:0/469834052 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34031600 con 0x7f4b3c0b3f20 2024-04-25T10:59:41.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.860+0000 7f4b33fff640 1 -- 172.21.15.69:0/469834052 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b340317a0 con 0x7f4b3c0b3f20 2024-04-25T10:59:41.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.860+0000 7f4b33fff640 1 -- 172.21.15.69:0/469834052 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34031b70 con 0x7f4b3c0b3f20 2024-04-25T10:59:41.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.861+0000 7f4b432af640 1 -- 172.21.15.69:0/469834052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b3c0b3f20 msgr2=0x7f4b3c0b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:41.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.861+0000 7f4b432af640 1 --2- 172.21.15.69:0/469834052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b3c0b3f20 0x7f4b3c0b4330 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009d90 tx=0x7f4b3402a5d0 comp rx=0 tx=0).stop 2024-04-25T10:59:41.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.861+0000 7f4b432af640 1 -- 172.21.15.69:0/469834052 shutdown_connections 2024-04-25T10:59:41.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.861+0000 7f4b432af640 1 --2- 172.21.15.69:0/469834052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b3c0b3f20 0x7f4b3c0b4330 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:41.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.861+0000 7f4b432af640 1 --2- 172.21.15.69:0/469834052 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b3c0cec90 0x7f4b3c0d2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:41.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.861+0000 7f4b432af640 1 -- 172.21.15.69:0/469834052 >> 172.21.15.69:0/469834052 conn(0x7f4b3c09b7d0 msgr2=0x7f4b3c09bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:41.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.861+0000 7f4b432af640 1 -- 172.21.15.69:0/469834052 shutdown_connections 2024-04-25T10:59:41.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.861+0000 7f4b432af640 1 -- 172.21.15.69:0/469834052 wait complete. 2024-04-25T10:59:41.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.863+0000 7f4b432af640 1 Processor -- start 2024-04-25T10:59:41.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.863+0000 7f4b432af640 1 -- start start 2024-04-25T10:59:41.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.864+0000 7f4b432af640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b3c0b3f20 0x7f4b3c160850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:41.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.864+0000 7f4b432af640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b3c0ce1c0 0x7f4b3c151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:41.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.864+0000 7f4b432af640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b3c0cec90 0x7f4b3c0c8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:41.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.864+0000 7f4b432af640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c0d31d0 con 0x7f4b3c0cec90 2024-04-25T10:59:41.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.864+0000 7f4b432af640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c0cf0a0 con 0x7f4b3c0b3f20 2024-04-25T10:59:41.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.864+0000 7f4b432af640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4b3c0b4870 con 0x7f4b3c0ce1c0 2024-04-25T10:59:41.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.864+0000 7f4b4184c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b3c0cec90 0x7f4b3c0c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:41.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.864+0000 7f4b4184c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b3c0cec90 0x7f4b3c0c8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43772/0 (socket says 172.21.15.69:43772) 2024-04-25T10:59:41.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.864+0000 7f4b4184c640 1 -- 172.21.15.69:0/895047161 learned_addr learned my addr 172.21.15.69:0/895047161 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:41.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.864+0000 7f4b4084a640 1 --2- 172.21.15.69:0/895047161 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b3c0ce1c0 0x7f4b3c151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:41.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.865+0000 7f4b4104b640 1 --2- 172.21.15.69:0/895047161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b3c0b3f20 0x7f4b3c160850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:41.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.865+0000 7f4b4184c640 1 -- 172.21.15.69:0/895047161 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b3c0ce1c0 msgr2=0x7f4b3c151ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:41.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.865+0000 7f4b4184c640 1 --2- 172.21.15.69:0/895047161 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b3c0ce1c0 0x7f4b3c151ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:41.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.865+0000 7f4b4184c640 1 -- 172.21.15.69:0/895047161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b3c0b3f20 msgr2=0x7f4b3c160850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:41.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.865+0000 7f4b4184c640 1 --2- 172.21.15.69:0/895047161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b3c0b3f20 0x7f4b3c160850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:41.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.865+0000 7f4b4184c640 1 -- 172.21.15.69:0/895047161 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b3c161080 con 0x7f4b3c0cec90 2024-04-25T10:59:41.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.865+0000 7f4b4084a640 1 --2- 172.21.15.69:0/895047161 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b3c0ce1c0 0x7f4b3c151ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:41.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.865+0000 7f4b4184c640 1 --2- 172.21.15.69:0/895047161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b3c0cec90 0x7f4b3c0c8770 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009d90 tx=0x7f4b34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:41.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.865+0000 7f4b31ffb640 1 -- 172.21.15.69:0/895047161 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34040070 con 0x7f4b3c0cec90 2024-04-25T10:59:41.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.865+0000 7f4b31ffb640 1 -- 172.21.15.69:0/895047161 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b34030b60 con 0x7f4b3c0cec90 2024-04-25T10:59:41.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.865+0000 7f4b432af640 1 -- 172.21.15.69:0/895047161 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b3c161360 con 0x7f4b3c0cec90 2024-04-25T10:59:41.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.865+0000 7f4b31ffb640 1 -- 172.21.15.69:0/895047161 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34009450 con 0x7f4b3c0cec90 2024-04-25T10:59:41.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.865+0000 7f4b432af640 1 -- 172.21.15.69:0/895047161 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b3c161920 con 0x7f4b3c0cec90 2024-04-25T10:59:41.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.866+0000 7f4b432af640 1 -- 172.21.15.69:0/895047161 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b3c050740 con 0x7f4b3c0cec90 2024-04-25T10:59:41.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.867+0000 7f4b31ffb640 1 -- 172.21.15.69:0/895047161 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4b34039040 con 0x7f4b3c0cec90 2024-04-25T10:59:41.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.867+0000 7f4b31ffb640 1 --2- 172.21.15.69:0/895047161 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4b200744e0 0x7f4b200769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:41.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.867+0000 7f4b31ffb640 1 -- 172.21.15.69:0/895047161 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4b340b6690 con 0x7f4b3c0cec90 2024-04-25T10:59:41.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.867+0000 7f4b4104b640 1 --2- 172.21.15.69:0/895047161 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4b200744e0 0x7f4b200769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:41.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.868+0000 7f4b4104b640 1 --2- 172.21.15.69:0/895047161 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4b200744e0 0x7f4b200769f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4b380010e0 tx=0x7f4b38009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:41.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:41.872+0000 7f4b31ffb640 1 -- 172.21.15.69:0/895047161 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4b34080d80 con 0x7f4b3c0cec90 2024-04-25T10:59:42.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.044+0000 7f4b432af640 1 -- 172.21.15.69:0/895047161 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4b3c160e10 con 0x7f4b3c0cec90 2024-04-25T10:59:42.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.044+0000 7f4b31ffb640 1 -- 172.21.15.69:0/895047161 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f4b34080680 con 0x7f4b3c0cec90 2024-04-25T10:59:42.045 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-25T10:59:42.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.046+0000 7f4b432af640 1 -- 172.21.15.69:0/895047161 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4b200744e0 msgr2=0x7f4b200769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:42.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.046+0000 7f4b432af640 1 --2- 172.21.15.69:0/895047161 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4b200744e0 0x7f4b200769f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4b380010e0 tx=0x7f4b38009040 comp rx=0 tx=0).stop 2024-04-25T10:59:42.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.046+0000 7f4b432af640 1 -- 172.21.15.69:0/895047161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b3c0cec90 msgr2=0x7f4b3c0c8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:42.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.046+0000 7f4b432af640 1 --2- 172.21.15.69:0/895047161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b3c0cec90 0x7f4b3c0c8770 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009d90 tx=0x7f4b34004a40 comp rx=0 tx=0).stop 2024-04-25T10:59:42.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.046+0000 7f4b432af640 1 -- 172.21.15.69:0/895047161 shutdown_connections 2024-04-25T10:59:42.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.046+0000 7f4b432af640 1 --2- 172.21.15.69:0/895047161 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4b200744e0 0x7f4b200769f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.046+0000 7f4b432af640 1 --2- 172.21.15.69:0/895047161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b3c0cec90 0x7f4b3c0c8770 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.046+0000 7f4b432af640 1 --2- 172.21.15.69:0/895047161 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b3c0ce1c0 0x7f4b3c151ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.046+0000 7f4b432af640 1 --2- 172.21.15.69:0/895047161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b3c0b3f20 0x7f4b3c160850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.046+0000 7f4b432af640 1 -- 172.21.15.69:0/895047161 >> 172.21.15.69:0/895047161 conn(0x7f4b3c09b7d0 msgr2=0x7f4b3c0702f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:42.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.046+0000 7f4b432af640 1 -- 172.21.15.69:0/895047161 shutdown_connections 2024-04-25T10:59:42.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.046+0000 7f4b432af640 1 -- 172.21.15.69:0/895047161 wait complete. 2024-04-25T10:59:42.058 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:59:42.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.164+0000 7f26825b2640 1 Processor -- start 2024-04-25T10:59:42.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.164+0000 7f26825b2640 1 -- start start 2024-04-25T10:59:42.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.166+0000 7f26825b2640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f267c0d1af0 0x7f267c0d1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:42.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.166+0000 7f26825b2640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f267c0b23e0 0x7f267c0b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:42.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.166+0000 7f26825b2640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f267c0cf4b0 con 0x7f267c161760 2024-04-25T10:59:42.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.166+0000 7f26825b2640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f267c0d00c0 con 0x7f267c0b23e0 2024-04-25T10:59:42.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.166+0000 7f26825b2640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f267c0cf790 con 0x7f267c0d1af0 2024-04-25T10:59:42.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.166+0000 7f2680b4f640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f267c0b23e0 0x7f267c0b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:42.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.166+0000 7f2680b4f640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f267c0b23e0 0x7f267c0b27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43786/0 (socket says 172.21.15.69:43786) 2024-04-25T10:59:42.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.166+0000 7f267bfff640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f267c161760 0x7f267c0b7d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:44304/0 (socket says 172.21.15.69:44304) 2024-04-25T10:59:42.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.166+0000 7f2680b4f640 1 -- 172.21.15.69:0/4134740016 learned_addr learned my addr 172.21.15.69:0/4134740016 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:42.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.166+0000 7f267b7fe640 1 --2- 172.21.15.69:0/4134740016 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f267c0d1af0 0x7f267c0d1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:42.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.166+0000 7f2680b4f640 1 -- 172.21.15.69:0/4134740016 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f267c0d1af0 msgr2=0x7f267c0d1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:42.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.166+0000 7f2680b4f640 1 --2- 172.21.15.69:0/4134740016 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f267c0d1af0 0x7f267c0d1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.166+0000 7f2680b4f640 1 -- 172.21.15.69:0/4134740016 >> v1:172.21.15.69:6789/0 conn(0x7f267c161760 legacy=0x7f267c0b7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:42.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.166+0000 7f2680b4f640 1 -- 172.21.15.69:0/4134740016 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267c0cfa70 con 0x7f267c0b23e0 2024-04-25T10:59:42.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.166+0000 7f267affd640 1 -- 172.21.15.69:0/4134740016 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2045508906 0 0) 0x7f267c0cf4b0 con 0x7f267c161760 2024-04-25T10:59:42.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.167+0000 7f2680b4f640 1 --2- 172.21.15.69:0/4134740016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f267c0b23e0 0x7f267c0b27f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2674009e50 tx=0x7f267402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8aed2f3d87fa7c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:42.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.167+0000 7f267affd640 1 -- 172.21.15.69:0/4134740016 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674036070 con 0x7f267c0b23e0 2024-04-25T10:59:42.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.167+0000 7f267affd640 1 -- 172.21.15.69:0/4134740016 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2674031600 con 0x7f267c0b23e0 2024-04-25T10:59:42.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.167+0000 7f267affd640 1 -- 172.21.15.69:0/4134740016 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26740319d0 con 0x7f267c0b23e0 2024-04-25T10:59:42.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.168+0000 7f26825b2640 1 -- 172.21.15.69:0/4134740016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f267c0b23e0 msgr2=0x7f267c0b27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:42.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.168+0000 7f26825b2640 1 --2- 172.21.15.69:0/4134740016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f267c0b23e0 0x7f267c0b27f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2674009e50 tx=0x7f267402a710 comp rx=0 tx=0).stop 2024-04-25T10:59:42.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.168+0000 7f26825b2640 1 -- 172.21.15.69:0/4134740016 shutdown_connections 2024-04-25T10:59:42.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.168+0000 7f26825b2640 1 --2- 172.21.15.69:0/4134740016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f267c0b23e0 0x7f267c0b27f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.168+0000 7f26825b2640 1 --2- 172.21.15.69:0/4134740016 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f267c0d1af0 0x7f267c0d1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.168+0000 7f26825b2640 1 -- 172.21.15.69:0/4134740016 >> 172.21.15.69:0/4134740016 conn(0x7f267c09b7d0 msgr2=0x7f267c09bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:42.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.168+0000 7f26825b2640 1 -- 172.21.15.69:0/4134740016 shutdown_connections 2024-04-25T10:59:42.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.168+0000 7f26825b2640 1 -- 172.21.15.69:0/4134740016 wait complete. 2024-04-25T10:59:42.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.169+0000 7f26825b2640 1 Processor -- start 2024-04-25T10:59:42.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.170+0000 7f26825b2640 1 -- start start 2024-04-25T10:59:42.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.171+0000 7f26825b2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f267c0b23e0 0x7f267c1f6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:42.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.171+0000 7f26825b2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f267c0d1af0 0x7f267c1fa580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:42.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.171+0000 7f26825b2640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f267c161760 0x7f267c1faac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:42.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.171+0000 7f26825b2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f267c0b84a0 con 0x7f267c0d1af0 2024-04-25T10:59:42.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.171+0000 7f26825b2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f267c161b70 con 0x7f267c0b23e0 2024-04-25T10:59:42.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.171+0000 7f26825b2640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f267c0b7940 con 0x7f267c161760 2024-04-25T10:59:42.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.171+0000 7f267b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f267c0d1af0 0x7f267c1fa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:42.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.171+0000 7f267b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f267c0d1af0 0x7f267c1fa580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43796/0 (socket says 172.21.15.69:43796) 2024-04-25T10:59:42.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.171+0000 7f267b7fe640 1 -- 172.21.15.69:0/1965190159 learned_addr learned my addr 172.21.15.69:0/1965190159 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:42.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.171+0000 7f2680b4f640 1 --2- 172.21.15.69:0/1965190159 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f267c161760 0x7f267c1faac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:42.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.171+0000 7f267bfff640 1 --2- 172.21.15.69:0/1965190159 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f267c0b23e0 0x7f267c1f6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:42.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.171+0000 7f267b7fe640 1 -- 172.21.15.69:0/1965190159 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f267c161760 msgr2=0x7f267c1faac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:42.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.171+0000 7f267b7fe640 1 --2- 172.21.15.69:0/1965190159 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f267c161760 0x7f267c1faac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.171+0000 7f267b7fe640 1 -- 172.21.15.69:0/1965190159 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f267c0b23e0 msgr2=0x7f267c1f6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:42.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.171+0000 7f267b7fe640 1 --2- 172.21.15.69:0/1965190159 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f267c0b23e0 0x7f267c1f6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.171+0000 7f267b7fe640 1 -- 172.21.15.69:0/1965190159 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267c1f71f0 con 0x7f267c0d1af0 2024-04-25T10:59:42.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.171+0000 7f2680b4f640 1 --2- 172.21.15.69:0/1965190159 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f267c161760 0x7f267c1faac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:42.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.171+0000 7f267bfff640 1 --2- 172.21.15.69:0/1965190159 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f267c0b23e0 0x7f267c1f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:42.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.172+0000 7f267b7fe640 1 --2- 172.21.15.69:0/1965190159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f267c0d1af0 0x7f267c1fa580 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f267000d5f0 tx=0x7f267000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:42.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.172+0000 7f2678ff9640 1 -- 172.21.15.69:0/1965190159 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26700125f0 con 0x7f267c0d1af0 2024-04-25T10:59:42.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.172+0000 7f2678ff9640 1 -- 172.21.15.69:0/1965190159 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2670019070 con 0x7f267c0d1af0 2024-04-25T10:59:42.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.172+0000 7f26825b2640 1 -- 172.21.15.69:0/1965190159 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f267c1f74d0 con 0x7f267c0d1af0 2024-04-25T10:59:42.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.172+0000 7f2678ff9640 1 -- 172.21.15.69:0/1965190159 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26700129d0 con 0x7f267c0d1af0 2024-04-25T10:59:42.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.172+0000 7f26825b2640 1 -- 172.21.15.69:0/1965190159 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f267c1f7e40 con 0x7f267c0d1af0 2024-04-25T10:59:42.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.173+0000 7f26825b2640 1 -- 172.21.15.69:0/1965190159 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f267c050740 con 0x7f267c0d1af0 2024-04-25T10:59:42.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.173+0000 7f2678ff9640 1 -- 172.21.15.69:0/1965190159 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2670012b70 con 0x7f267c0d1af0 2024-04-25T10:59:42.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.174+0000 7f2678ff9640 1 --2- 172.21.15.69:0/1965190159 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f26680744e0 0x7f26680769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:42.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.174+0000 7f267bfff640 1 --2- 172.21.15.69:0/1965190159 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f26680744e0 0x7f26680769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:42.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.174+0000 7f2678ff9640 1 -- 172.21.15.69:0/1965190159 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f26700955c0 con 0x7f267c0d1af0 2024-04-25T10:59:42.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.174+0000 7f267bfff640 1 --2- 172.21.15.69:0/1965190159 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f26680744e0 0x7f26680769f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f266c000f60 tx=0x7f266c007cc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:42.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.179+0000 7f2678ff9640 1 -- 172.21.15.69:0/1965190159 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f267005fd50 con 0x7f267c0d1af0 2024-04-25T10:59:42.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.355+0000 7f26825b2640 1 -- 172.21.15.69:0/1965190159 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f267c0b21c0 con 0x7f267c0d1af0 2024-04-25T10:59:42.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.356+0000 7f2678ff9640 1 -- 172.21.15.69:0/1965190159 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f267005f650 con 0x7f267c0d1af0 2024-04-25T10:59:42.357 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:42.357 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":20,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T10:59:20.630151+0000","last_up_change":"2024-04-25T10:59:05.546839+0000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:59:06.422674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:59:16.727980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:59:17.938486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6825","nonce":1876101065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6827","nonce":1876101065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6831","nonce":1876101065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6829","nonce":1876101065}]},"public_addr":"172.21.15.69:6825/1876101065","cluster_addr":"172.21.15.69:6827/1876101065","heartbeat_back_addr":"172.21.15.69:6831/1876101065","heartbeat_front_addr":"172.21.15.69:6829/1876101065","state":["exists","up"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6801","nonce":1806438563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6803","nonce":1806438563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6807","nonce":1806438563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6805","nonce":1806438563}]},"public_addr":"172.21.15.69:6801/1806438563","cluster_addr":"172.21.15.69:6803/1806438563","heartbeat_back_addr":"172.21.15.69:6807/1806438563","heartbeat_front_addr":"172.21.15.69:6805/1806438563","state":["exists","up"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6817","nonce":1593617983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6819","nonce":1593617983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6823","nonce":1593617983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6821","nonce":1593617983}]},"public_addr":"172.21.15.69:6817/1593617983","cluster_addr":"172.21.15.69:6819/1593617983","heartbeat_back_addr":"172.21.15.69:6823/1593617983","heartbeat_front_addr":"172.21.15.69:6821/1593617983","state":["exists","up"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6809","nonce":3950661294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6811","nonce":3950661294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6815","nonce":3950661294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6813","nonce":3950661294}]},"public_addr":"172.21.15.69:6809/3950661294","cluster_addr":"172.21.15.69:6811/3950661294","heartbeat_back_addr":"172.21.15.69:6815/3950661294","heartbeat_front_addr":"172.21.15.69:6813/3950661294","state":["exists","up"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6801","nonce":2508929872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6803","nonce":2508929872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6807","nonce":2508929872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6805","nonce":2508929872}]},"public_addr":"172.21.15.146:6801/2508929872","cluster_addr":"172.21.15.146:6803/2508929872","heartbeat_back_addr":"172.21.15.146:6807/2508929872","heartbeat_front_addr":"172.21.15.146:6805/2508929872","state":["exists","up"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6811","nonce":1310911138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6816","nonce":1310911138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6823","nonce":1310911138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6821","nonce":1310911138}]},"public_addr":"172.21.15.146:6811/1310911138","cluster_addr":"172.21.15.146:6816/1310911138","heartbeat_back_addr":"172.21.15.146:6823/1310911138","heartbeat_front_addr":"172.21.15.146:6821/1310911138","state":["exists","up"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6810","nonce":634688021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6813","nonce":634688021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6820","nonce":634688021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6817","nonce":634688021}]},"public_addr":"172.21.15.146:6810/634688021","cluster_addr":"172.21.15.146:6813/634688021","heartbeat_back_addr":"172.21.15.146:6820/634688021","heartbeat_front_addr":"172.21.15.146:6817/634688021","state":["exists","up"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6825","nonce":3814215935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6827","nonce":3814215935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6831","nonce":3814215935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6829","nonce":3814215935}]},"public_addr":"172.21.15.146:6825/3814215935","cluster_addr":"172.21.15.146:6827/3814215935","heartbeat_back_addr":"172.21.15.146:6831/3814215935","heartbeat_front_addr":"172.21.15.146:6829/3814215935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.820200+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.850189+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.764560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.866248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.807897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.869881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.782310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.827131+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:59:42.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.358+0000 7f26825b2640 1 -- 172.21.15.69:0/1965190159 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f26680744e0 msgr2=0x7f26680769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:42.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.358+0000 7f26825b2640 1 --2- 172.21.15.69:0/1965190159 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f26680744e0 0x7f26680769f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f266c000f60 tx=0x7f266c007cc0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.358+0000 7f26825b2640 1 -- 172.21.15.69:0/1965190159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f267c0d1af0 msgr2=0x7f267c1fa580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:42.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.358+0000 7f26825b2640 1 --2- 172.21.15.69:0/1965190159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f267c0d1af0 0x7f267c1fa580 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f267000d5f0 tx=0x7f267000dab0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.359+0000 7f26825b2640 1 -- 172.21.15.69:0/1965190159 shutdown_connections 2024-04-25T10:59:42.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.359+0000 7f26825b2640 1 --2- 172.21.15.69:0/1965190159 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f26680744e0 0x7f26680769f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.359+0000 7f26825b2640 1 --2- 172.21.15.69:0/1965190159 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f267c161760 0x7f267c1faac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.359+0000 7f26825b2640 1 --2- 172.21.15.69:0/1965190159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f267c0d1af0 0x7f267c1fa580 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.359+0000 7f26825b2640 1 --2- 172.21.15.69:0/1965190159 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f267c0b23e0 0x7f267c1f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.359+0000 7f26825b2640 1 -- 172.21.15.69:0/1965190159 >> 172.21.15.69:0/1965190159 conn(0x7f267c09b7d0 msgr2=0x7f267c0b3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:42.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.359+0000 7f26825b2640 1 -- 172.21.15.69:0/1965190159 shutdown_connections 2024-04-25T10:59:42.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.359+0000 7f26825b2640 1 -- 172.21.15.69:0/1965190159 wait complete. 2024-04-25T10:59:42.371 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:59:06.422674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:59:16.727980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:59:17.938486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:59:42.371 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:59:42.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.481+0000 7f7307492640 1 Processor -- start 2024-04-25T10:59:42.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.481+0000 7f7307492640 1 -- start start 2024-04-25T10:59:42.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.483+0000 7f7307492640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f73000d1e20 0x7f73000d3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:42.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.483+0000 7f7307492640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7300164020 0x7f7300164430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:42.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.483+0000 7f7307492640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7300164970 con 0x7f73000b2e60 2024-04-25T10:59:42.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.483+0000 7f7307492640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f73000d23f0 con 0x7f73000d1e20 2024-04-25T10:59:42.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.483+0000 7f7307492640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f73000d26d0 con 0x7f7300164020 2024-04-25T10:59:42.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.483+0000 7f730522e640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f73000d1e20 0x7f73000d3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:42.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.483+0000 7f7305a2f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7300164020 0x7f7300164430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:42.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.483+0000 7f730522e640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f73000d1e20 0x7f73000d3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.69:49748/0 (socket says 172.21.15.69:49748) 2024-04-25T10:59:42.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.483+0000 7f730522e640 1 -- 172.21.15.69:0/452411439 learned_addr learned my addr 172.21.15.69:0/452411439 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:42.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.483+0000 7f730522e640 1 -- 172.21.15.69:0/452411439 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7300164020 msgr2=0x7f7300164430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:42.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.483+0000 7f730522e640 1 --2- 172.21.15.69:0/452411439 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7300164020 0x7f7300164430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.483+0000 7f730522e640 1 -- 172.21.15.69:0/452411439 >> v1:172.21.15.146:6789/0 conn(0x7f73000b2e60 legacy=0x7f730008d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:42.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.483+0000 7f730522e640 1 -- 172.21.15.69:0/452411439 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73000d29b0 con 0x7f73000d1e20 2024-04-25T10:59:42.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.483+0000 7f72f7fff640 1 -- 172.21.15.69:0/452411439 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1386880094 0 0) 0x7f72f0001110 con 0x7f73000b2e60 2024-04-25T10:59:42.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.483+0000 7f7305a2f640 1 --2- 172.21.15.69:0/452411439 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7300164020 0x7f7300164430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:59:42.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.484+0000 7f730522e640 1 --2- 172.21.15.69:0/452411439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73000d1e20 0x7f73000d3490 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f72fc009870 tx=0x7f72fc02a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95621407de9c4716 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:42.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.484+0000 7f72f7fff640 1 -- 172.21.15.69:0/452411439 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc037070 con 0x7f73000d1e20 2024-04-25T10:59:42.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.484+0000 7f72f7fff640 1 -- 172.21.15.69:0/452411439 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72fc02ae50 con 0x7f73000d1e20 2024-04-25T10:59:42.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.484+0000 7f72f7fff640 1 -- 172.21.15.69:0/452411439 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc032860 con 0x7f73000d1e20 2024-04-25T10:59:42.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.485+0000 7f7307492640 1 -- 172.21.15.69:0/452411439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73000d1e20 msgr2=0x7f73000d3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:42.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.485+0000 7f7307492640 1 --2- 172.21.15.69:0/452411439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73000d1e20 0x7f73000d3490 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f72fc009870 tx=0x7f72fc02a300 comp rx=0 tx=0).stop 2024-04-25T10:59:42.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.485+0000 7f7307492640 1 -- 172.21.15.69:0/452411439 shutdown_connections 2024-04-25T10:59:42.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.485+0000 7f7307492640 1 --2- 172.21.15.69:0/452411439 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7300164020 0x7f7300164430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.485+0000 7f7307492640 1 --2- 172.21.15.69:0/452411439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73000d1e20 0x7f73000d3490 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.485+0000 7f7307492640 1 -- 172.21.15.69:0/452411439 >> 172.21.15.69:0/452411439 conn(0x7f73000d0530 msgr2=0x7f73000d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:42.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.485+0000 7f7307492640 1 -- 172.21.15.69:0/452411439 shutdown_connections 2024-04-25T10:59:42.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.485+0000 7f7307492640 1 -- 172.21.15.69:0/452411439 wait complete. 2024-04-25T10:59:42.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.487+0000 7f7307492640 1 Processor -- start 2024-04-25T10:59:42.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.487+0000 7f7307492640 1 -- start start 2024-04-25T10:59:42.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.488+0000 7f7307492640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73000b2e60 0x7f73000cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:42.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.488+0000 7f7307492640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73000d1e20 0x7f730014eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:42.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.488+0000 7f7307492640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7300164020 0x7f73001fa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:42.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.488+0000 7f7307492640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f73000ce070 con 0x7f7300164020 2024-04-25T10:59:42.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.488+0000 7f7307492640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f73000d2f70 con 0x7f73000b2e60 2024-04-25T10:59:42.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.488+0000 7f7307492640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f73000ce1f0 con 0x7f73000d1e20 2024-04-25T10:59:42.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.488+0000 7f7305a2f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7300164020 0x7f73001fa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:42.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.488+0000 7f7305a2f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7300164020 0x7f73001fa6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43804/0 (socket says 172.21.15.69:43804) 2024-04-25T10:59:42.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.488+0000 7f7305a2f640 1 -- 172.21.15.69:0/804679957 learned_addr learned my addr 172.21.15.69:0/804679957 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:42.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.489+0000 7f730522e640 1 --2- 172.21.15.69:0/804679957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73000b2e60 0x7f73000cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:42.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.489+0000 7f7304a2d640 1 --2- 172.21.15.69:0/804679957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73000d1e20 0x7f730014eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:42.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.489+0000 7f7305a2f640 1 -- 172.21.15.69:0/804679957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73000d1e20 msgr2=0x7f730014eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:42.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.489+0000 7f7305a2f640 1 --2- 172.21.15.69:0/804679957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73000d1e20 0x7f730014eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.489+0000 7f7305a2f640 1 -- 172.21.15.69:0/804679957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73000b2e60 msgr2=0x7f73000cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:42.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.489+0000 7f7305a2f640 1 --2- 172.21.15.69:0/804679957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73000b2e60 0x7f73000cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.489+0000 7f7305a2f640 1 -- 172.21.15.69:0/804679957 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73000c99b0 con 0x7f7300164020 2024-04-25T10:59:42.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.489+0000 7f7304a2d640 1 --2- 172.21.15.69:0/804679957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73000d1e20 0x7f730014eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:42.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.489+0000 7f730522e640 1 --2- 172.21.15.69:0/804679957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73000b2e60 0x7f73000cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:42.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.489+0000 7f7305a2f640 1 --2- 172.21.15.69:0/804679957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7300164020 0x7f73001fa6c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f72f800efc0 tx=0x7f72f800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:42.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.489+0000 7f72f5ffb640 1 -- 172.21.15.69:0/804679957 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f8018030 con 0x7f7300164020 2024-04-25T10:59:42.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.489+0000 7f72f5ffb640 1 -- 172.21.15.69:0/804679957 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72f800cd30 con 0x7f7300164020 2024-04-25T10:59:42.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.489+0000 7f7307492640 1 -- 172.21.15.69:0/804679957 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73000c9c90 con 0x7f7300164020 2024-04-25T10:59:42.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.489+0000 7f72f5ffb640 1 -- 172.21.15.69:0/804679957 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f8003bb0 con 0x7f7300164020 2024-04-25T10:59:42.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.489+0000 7f7307492640 1 -- 172.21.15.69:0/804679957 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73000ca250 con 0x7f7300164020 2024-04-25T10:59:42.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.490+0000 7f7307492640 1 -- 172.21.15.69:0/804679957 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f730015e710 con 0x7f7300164020 2024-04-25T10:59:42.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.491+0000 7f72f5ffb640 1 -- 172.21.15.69:0/804679957 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f72f801b070 con 0x7f7300164020 2024-04-25T10:59:42.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.491+0000 7f72f5ffb640 1 --2- 172.21.15.69:0/804679957 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f72e40744b0 0x7f72e40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:42.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.491+0000 7f72f5ffb640 1 -- 172.21.15.69:0/804679957 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f72f8095bb0 con 0x7f7300164020 2024-04-25T10:59:42.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.491+0000 7f730522e640 1 --2- 172.21.15.69:0/804679957 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f72e40744b0 0x7f72e40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:42.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.492+0000 7f730522e640 1 --2- 172.21.15.69:0/804679957 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f72e40744b0 0x7f72e40769c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f72fc034000 tx=0x7f72fc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:42.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.496+0000 7f72f5ffb640 1 -- 172.21.15.69:0/804679957 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f72f8010040 con 0x7f7300164020 2024-04-25T10:59:42.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.671+0000 7f7307492640 1 -- 172.21.15.69:0/804679957 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7300050740 con 0x7f7300164020 2024-04-25T10:59:42.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.672+0000 7f72f5ffb640 1 -- 172.21.15.69:0/804679957 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f72f8060350 con 0x7f7300164020 2024-04-25T10:59:42.673 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-25T10:59:42.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.674+0000 7f7307492640 1 -- 172.21.15.69:0/804679957 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f72e40744b0 msgr2=0x7f72e40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:42.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.674+0000 7f7307492640 1 --2- 172.21.15.69:0/804679957 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f72e40744b0 0x7f72e40769c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f72fc034000 tx=0x7f72fc02e000 comp rx=0 tx=0).stop 2024-04-25T10:59:42.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.674+0000 7f7307492640 1 -- 172.21.15.69:0/804679957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7300164020 msgr2=0x7f73001fa6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:42.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.674+0000 7f7307492640 1 --2- 172.21.15.69:0/804679957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7300164020 0x7f73001fa6c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f72f800efc0 tx=0x7f72f800c490 comp rx=0 tx=0).stop 2024-04-25T10:59:42.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.674+0000 7f7307492640 1 -- 172.21.15.69:0/804679957 shutdown_connections 2024-04-25T10:59:42.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.674+0000 7f7307492640 1 --2- 172.21.15.69:0/804679957 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f72e40744b0 0x7f72e40769c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.674+0000 7f7307492640 1 --2- 172.21.15.69:0/804679957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7300164020 0x7f73001fa6c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.674+0000 7f7307492640 1 --2- 172.21.15.69:0/804679957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73000d1e20 0x7f730014eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.674+0000 7f7307492640 1 --2- 172.21.15.69:0/804679957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73000b2e60 0x7f73000cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.674+0000 7f7307492640 1 -- 172.21.15.69:0/804679957 >> 172.21.15.69:0/804679957 conn(0x7f73000d0530 msgr2=0x7f730006d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:42.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.674+0000 7f7307492640 1 -- 172.21.15.69:0/804679957 shutdown_connections 2024-04-25T10:59:42.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.674+0000 7f7307492640 1 -- 172.21.15.69:0/804679957 wait complete. 2024-04-25T10:59:42.686 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:59:42.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.796+0000 7fa92f538640 1 Processor -- start 2024-04-25T10:59:42.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.796+0000 7fa92f538640 1 -- start start 2024-04-25T10:59:42.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.798+0000 7fa92f538640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fa928070630 0x7fa9280cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:42.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.798+0000 7fa92f538640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa92815adf0 0x7fa92806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:42.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.798+0000 7fa92f538640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa92806df20 con 0x7fa92806db10 2024-04-25T10:59:42.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.798+0000 7fa92f538640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fa9280713f0 con 0x7fa928070630 2024-04-25T10:59:42.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.798+0000 7fa92f538640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa9280701a0 con 0x7fa92815adf0 2024-04-25T10:59:42.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.798+0000 7fa92d2d4640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fa928070630 0x7fa9280cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:42.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.798+0000 7fa92d2d4640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fa928070630 0x7fa9280cec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43816/0 (socket says 172.21.15.69:43816) 2024-04-25T10:59:42.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.798+0000 7fa92d2d4640 1 -- 172.21.15.69:0/3393899444 learned_addr learned my addr 172.21.15.69:0/3393899444 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:42.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.798+0000 7fa92cad3640 1 --2- 172.21.15.69:0/3393899444 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa92815adf0 0x7fa92806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:42.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.798+0000 7fa92d2d4640 1 -- 172.21.15.69:0/3393899444 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa92815adf0 msgr2=0x7fa92806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:42.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.798+0000 7fa92d2d4640 1 --2- 172.21.15.69:0/3393899444 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa92815adf0 0x7fa92806d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.798+0000 7fa92d2d4640 1 -- 172.21.15.69:0/3393899444 >> v1:172.21.15.146:6789/0 conn(0x7fa92806db10 legacy=0x7fa9280b59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:42.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.798+0000 7fa92d2d4640 1 -- 172.21.15.69:0/3393899444 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa928071110 con 0x7fa928070630 2024-04-25T10:59:42.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.799+0000 7fa92d2d4640 1 --2- 172.21.15.69:0/3393899444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa928070630 0x7fa9280cec50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa92400d150 tx=0x7fa92402c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f2f11c619d56ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:42.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.799+0000 7fa91ffff640 1 -- 172.21.15.69:0/3393899444 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa924039070 con 0x7fa928070630 2024-04-25T10:59:42.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.799+0000 7fa91ffff640 1 -- 172.21.15.69:0/3393899444 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa924009600 con 0x7fa928070630 2024-04-25T10:59:42.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.799+0000 7fa91ffff640 1 -- 172.21.15.69:0/3393899444 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9240099d0 con 0x7fa928070630 2024-04-25T10:59:42.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.800+0000 7fa92f538640 1 -- 172.21.15.69:0/3393899444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa928070630 msgr2=0x7fa9280cec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:42.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.800+0000 7fa92f538640 1 --2- 172.21.15.69:0/3393899444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa928070630 0x7fa9280cec50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa92400d150 tx=0x7fa92402c840 comp rx=0 tx=0).stop 2024-04-25T10:59:42.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.800+0000 7fa92f538640 1 -- 172.21.15.69:0/3393899444 shutdown_connections 2024-04-25T10:59:42.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.800+0000 7fa92f538640 1 --2- 172.21.15.69:0/3393899444 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa92815adf0 0x7fa92806d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.800+0000 7fa92f538640 1 --2- 172.21.15.69:0/3393899444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa928070630 0x7fa9280cec50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.800+0000 7fa92f538640 1 -- 172.21.15.69:0/3393899444 >> 172.21.15.69:0/3393899444 conn(0x7fa928073090 msgr2=0x7fa9280734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:42.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.800+0000 7fa92f538640 1 -- 172.21.15.69:0/3393899444 shutdown_connections 2024-04-25T10:59:42.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.800+0000 7fa92f538640 1 -- 172.21.15.69:0/3393899444 wait complete. 2024-04-25T10:59:42.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.801+0000 7fa92f538640 1 Processor -- start 2024-04-25T10:59:42.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.801+0000 7fa92f538640 1 -- start start 2024-04-25T10:59:42.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.803+0000 7fa92f538640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa92806db10 0x7fa928152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:42.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.803+0000 7fa92f538640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa928070630 0x7fa9281f2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:42.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.803+0000 7fa92f538640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa92815adf0 0x7fa9281f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:42.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.803+0000 7fa92f538640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa9280b0d00 con 0x7fa928070630 2024-04-25T10:59:42.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.803+0000 7fa92f538640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa92806fe30 con 0x7fa92815adf0 2024-04-25T10:59:42.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.803+0000 7fa92f538640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa9280b0e80 con 0x7fa92806db10 2024-04-25T10:59:42.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.803+0000 7fa92cad3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa928070630 0x7fa9281f2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:42.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.803+0000 7fa92cad3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa928070630 0x7fa9281f2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43818/0 (socket says 172.21.15.69:43818) 2024-04-25T10:59:42.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.803+0000 7fa92cad3640 1 -- 172.21.15.69:0/698170362 learned_addr learned my addr 172.21.15.69:0/698170362 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:42.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.803+0000 7fa92d2d4640 1 --2- 172.21.15.69:0/698170362 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa92806db10 0x7fa928152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:42.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.803+0000 7fa92dad5640 1 --2- 172.21.15.69:0/698170362 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa92815adf0 0x7fa9281f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:42.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.803+0000 7fa92cad3640 1 -- 172.21.15.69:0/698170362 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa92806db10 msgr2=0x7fa928152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:42.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.803+0000 7fa92cad3640 1 --2- 172.21.15.69:0/698170362 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa92806db10 0x7fa928152750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.803+0000 7fa92cad3640 1 -- 172.21.15.69:0/698170362 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa92815adf0 msgr2=0x7fa9281f30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:42.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.803+0000 7fa92cad3640 1 --2- 172.21.15.69:0/698170362 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa92815adf0 0x7fa9281f30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.803+0000 7fa92cad3640 1 -- 172.21.15.69:0/698170362 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92815bf80 con 0x7fa928070630 2024-04-25T10:59:42.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.803+0000 7fa92d2d4640 1 --2- 172.21.15.69:0/698170362 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa92806db10 0x7fa928152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:42.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.803+0000 7fa92dad5640 1 --2- 172.21.15.69:0/698170362 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa92815adf0 0x7fa9281f30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:42.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.803+0000 7fa92cad3640 1 --2- 172.21.15.69:0/698170362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa928070630 0x7fa9281f2ba0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa91800d620 tx=0x7fa91800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:42.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.803+0000 7fa91dffb640 1 -- 172.21.15.69:0/698170362 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91800b620 con 0x7fa928070630 2024-04-25T10:59:42.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.803+0000 7fa91dffb640 1 -- 172.21.15.69:0/698170362 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa918019070 con 0x7fa928070630 2024-04-25T10:59:42.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.803+0000 7fa92f538640 1 -- 172.21.15.69:0/698170362 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa92815c260 con 0x7fa928070630 2024-04-25T10:59:42.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.803+0000 7fa91dffb640 1 -- 172.21.15.69:0/698170362 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91800ba00 con 0x7fa928070630 2024-04-25T10:59:42.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.804+0000 7fa92f538640 1 -- 172.21.15.69:0/698170362 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa92815c820 con 0x7fa928070630 2024-04-25T10:59:42.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.805+0000 7fa92f538640 1 -- 172.21.15.69:0/698170362 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa928050740 con 0x7fa928070630 2024-04-25T10:59:42.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.805+0000 7fa91dffb640 1 -- 172.21.15.69:0/698170362 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa91800bba0 con 0x7fa928070630 2024-04-25T10:59:42.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.806+0000 7fa91dffb640 1 --2- 172.21.15.69:0/698170362 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa90c0744b0 0x7fa90c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:42.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.806+0000 7fa92d2d4640 1 --2- 172.21.15.69:0/698170362 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa90c0744b0 0x7fa90c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:42.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.806+0000 7fa91dffb640 1 -- 172.21.15.69:0/698170362 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa9180955e0 con 0x7fa928070630 2024-04-25T10:59:42.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.806+0000 7fa92d2d4640 1 --2- 172.21.15.69:0/698170362 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa90c0744b0 0x7fa90c0769c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa924036000 tx=0x7fa92400b000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:42.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.810+0000 7fa91dffb640 1 -- 172.21.15.69:0/698170362 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa91805fe00 con 0x7fa928070630 2024-04-25T10:59:42.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.983+0000 7fa92f538640 1 -- 172.21.15.69:0/698170362 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa928073c80 con 0x7fa928070630 2024-04-25T10:59:42.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.984+0000 7fa91dffb640 1 -- 172.21.15.69:0/698170362 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fa91805f700 con 0x7fa928070630 2024-04-25T10:59:42.985 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-25T10:59:42.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.986+0000 7fa92f538640 1 -- 172.21.15.69:0/698170362 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa90c0744b0 msgr2=0x7fa90c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:42.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.986+0000 7fa92f538640 1 --2- 172.21.15.69:0/698170362 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa90c0744b0 0x7fa90c0769c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa924036000 tx=0x7fa92400b000 comp rx=0 tx=0).stop 2024-04-25T10:59:42.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.986+0000 7fa92f538640 1 -- 172.21.15.69:0/698170362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa928070630 msgr2=0x7fa9281f2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:42.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.986+0000 7fa92f538640 1 --2- 172.21.15.69:0/698170362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa928070630 0x7fa9281f2ba0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa91800d620 tx=0x7fa91800dae0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.986+0000 7fa92f538640 1 -- 172.21.15.69:0/698170362 shutdown_connections 2024-04-25T10:59:42.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.986+0000 7fa92f538640 1 --2- 172.21.15.69:0/698170362 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa90c0744b0 0x7fa90c0769c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.986+0000 7fa92f538640 1 --2- 172.21.15.69:0/698170362 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa92815adf0 0x7fa9281f30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.986+0000 7fa92f538640 1 --2- 172.21.15.69:0/698170362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa928070630 0x7fa9281f2ba0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.986+0000 7fa92f538640 1 --2- 172.21.15.69:0/698170362 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa92806db10 0x7fa928152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:42.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.986+0000 7fa92f538640 1 -- 172.21.15.69:0/698170362 >> 172.21.15.69:0/698170362 conn(0x7fa928073090 msgr2=0x7fa9280b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:42.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.986+0000 7fa92f538640 1 -- 172.21.15.69:0/698170362 shutdown_connections 2024-04-25T10:59:42.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:42.986+0000 7fa92f538640 1 -- 172.21.15.69:0/698170362 wait complete. 2024-04-25T10:59:42.997 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:59:43.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.109+0000 7fa34b042640 1 Processor -- start 2024-04-25T10:59:43.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.109+0000 7fa34b042640 1 -- start start 2024-04-25T10:59:43.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.111+0000 7fa34b042640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa344161a50 0x7fa3440b1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:43.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.111+0000 7fa34b042640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fa3440b17c0 0x7fa34406d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:43.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.111+0000 7fa34b042640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3440b1bd0 con 0x7fa34406dba0 2024-04-25T10:59:43.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.111+0000 7fa34b042640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fa34406dfb0 con 0x7fa3440b17c0 2024-04-25T10:59:43.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.111+0000 7fa34b042640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa3440b3250 con 0x7fa344161a50 2024-04-25T10:59:43.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.111+0000 7fa343fff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fa3440b17c0 0x7fa34406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:43.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.111+0000 7fa343fff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fa3440b17c0 0x7fa34406d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43826/0 (socket says 172.21.15.69:43826) 2024-04-25T10:59:43.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.111+0000 7fa343fff640 1 -- 172.21.15.69:0/2154712670 learned_addr learned my addr 172.21.15.69:0/2154712670 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:43.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.111+0000 7fa348dde640 1 --2- 172.21.15.69:0/2154712670 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa344161a50 0x7fa3440b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:43.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.111+0000 7fa343fff640 1 -- 172.21.15.69:0/2154712670 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa344161a50 msgr2=0x7fa3440b1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:43.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.111+0000 7fa343fff640 1 --2- 172.21.15.69:0/2154712670 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa344161a50 0x7fa3440b1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:43.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.111+0000 7fa343fff640 1 -- 172.21.15.69:0/2154712670 >> v1:172.21.15.146:6789/0 conn(0x7fa34406dba0 legacy=0x7fa34406c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:43.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.111+0000 7fa343fff640 1 -- 172.21.15.69:0/2154712670 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3440b3530 con 0x7fa3440b17c0 2024-04-25T10:59:43.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.111+0000 7fa348dde640 1 --2- 172.21.15.69:0/2154712670 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa344161a50 0x7fa3440b1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:43.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.112+0000 7fa343fff640 1 --2- 172.21.15.69:0/2154712670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3440b17c0 0x7fa34406d660 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa338009870 tx=0x7fa33802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625b1e448ed57a6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:43.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.112+0000 7fa3437fe640 1 -- 172.21.15.69:0/2154712670 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa338037070 con 0x7fa3440b17c0 2024-04-25T10:59:43.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.112+0000 7fa3437fe640 1 -- 172.21.15.69:0/2154712670 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa338032440 con 0x7fa3440b17c0 2024-04-25T10:59:43.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.112+0000 7fa3437fe640 1 -- 172.21.15.69:0/2154712670 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa338032810 con 0x7fa3440b17c0 2024-04-25T10:59:43.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.113+0000 7fa34b042640 1 -- 172.21.15.69:0/2154712670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3440b17c0 msgr2=0x7fa34406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:43.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.113+0000 7fa34b042640 1 --2- 172.21.15.69:0/2154712670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3440b17c0 0x7fa34406d660 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa338009870 tx=0x7fa33802a1c0 comp rx=0 tx=0).stop 2024-04-25T10:59:43.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.113+0000 7fa34b042640 1 -- 172.21.15.69:0/2154712670 shutdown_connections 2024-04-25T10:59:43.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.113+0000 7fa34b042640 1 --2- 172.21.15.69:0/2154712670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3440b17c0 0x7fa34406d660 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:43.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.113+0000 7fa34b042640 1 --2- 172.21.15.69:0/2154712670 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa344161a50 0x7fa3440b1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:43.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.113+0000 7fa34b042640 1 -- 172.21.15.69:0/2154712670 >> 172.21.15.69:0/2154712670 conn(0x7fa34409b7d0 msgr2=0x7fa3440b70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:43.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.113+0000 7fa34b042640 1 -- 172.21.15.69:0/2154712670 shutdown_connections 2024-04-25T10:59:43.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.113+0000 7fa34b042640 1 -- 172.21.15.69:0/2154712670 wait complete. 2024-04-25T10:59:43.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.115+0000 7fa34b042640 1 Processor -- start 2024-04-25T10:59:43.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.115+0000 7fa34b042640 1 -- start start 2024-04-25T10:59:43.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.116+0000 7fa34b042640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa34406dba0 0x7fa34414d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:43.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.116+0000 7fa34b042640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3440b17c0 0x7fa34409cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:43.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.116+0000 7fa34b042640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa344161a50 0x7fa34415f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:43.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.116+0000 7fa34b042640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa3440af050 con 0x7fa3440b17c0 2024-04-25T10:59:43.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.116+0000 7fa34b042640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa3440b3af0 con 0x7fa344161a50 2024-04-25T10:59:43.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.116+0000 7fa34b042640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa344161e60 con 0x7fa34406dba0 2024-04-25T10:59:43.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.116+0000 7fa343fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3440b17c0 0x7fa34409cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:43.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.116+0000 7fa343fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3440b17c0 0x7fa34409cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43836/0 (socket says 172.21.15.69:43836) 2024-04-25T10:59:43.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.116+0000 7fa343fff640 1 -- 172.21.15.69:0/3644316452 learned_addr learned my addr 172.21.15.69:0/3644316452 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:43.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.116+0000 7fa343fff640 1 -- 172.21.15.69:0/3644316452 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa34406dba0 msgr2=0x7fa34414d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:43.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.116+0000 7fa343fff640 1 --2- 172.21.15.69:0/3644316452 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa34406dba0 0x7fa34414d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:43.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.116+0000 7fa343fff640 1 -- 172.21.15.69:0/3644316452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa344161a50 msgr2=0x7fa34415f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:43.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.116+0000 7fa343fff640 1 --2- 172.21.15.69:0/3644316452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa344161a50 0x7fa34415f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:43.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.116+0000 7fa343fff640 1 -- 172.21.15.69:0/3644316452 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa34409d7e0 con 0x7fa3440b17c0 2024-04-25T10:59:43.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.117+0000 7fa343fff640 1 --2- 172.21.15.69:0/3644316452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3440b17c0 0x7fa34409cd30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa338009990 tx=0x7fa3380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:43.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.117+0000 7fa3417fa640 1 -- 172.21.15.69:0/3644316452 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa338041020 con 0x7fa3440b17c0 2024-04-25T10:59:43.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.117+0000 7fa3417fa640 1 -- 172.21.15.69:0/3644316452 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa338030800 con 0x7fa3440b17c0 2024-04-25T10:59:43.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.117+0000 7fa34b042640 1 -- 172.21.15.69:0/3644316452 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa34409dce0 con 0x7fa3440b17c0 2024-04-25T10:59:43.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.117+0000 7fa3417fa640 1 -- 172.21.15.69:0/3644316452 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa338037040 con 0x7fa3440b17c0 2024-04-25T10:59:43.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.117+0000 7fa34b042640 1 -- 172.21.15.69:0/3644316452 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa34409e2a0 con 0x7fa3440b17c0 2024-04-25T10:59:43.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.118+0000 7fa34b042640 1 -- 172.21.15.69:0/3644316452 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa344050740 con 0x7fa3440b17c0 2024-04-25T10:59:43.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.119+0000 7fa3417fa640 1 -- 172.21.15.69:0/3644316452 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa338044020 con 0x7fa3440b17c0 2024-04-25T10:59:43.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.119+0000 7fa3417fa640 1 --2- 172.21.15.69:0/3644316452 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa3300744e0 0x7fa3300769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:43.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.119+0000 7fa348dde640 1 --2- 172.21.15.69:0/3644316452 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa3300744e0 0x7fa3300769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:43.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.119+0000 7fa3417fa640 1 -- 172.21.15.69:0/3644316452 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa3380b5a80 con 0x7fa3440b17c0 2024-04-25T10:59:43.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.120+0000 7fa348dde640 1 --2- 172.21.15.69:0/3644316452 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa3300744e0 0x7fa3300769f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa334006fd0 tx=0x7fa334008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:43.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.124+0000 7fa3417fa640 1 -- 172.21.15.69:0/3644316452 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa33804b030 con 0x7fa3440b17c0 2024-04-25T10:59:43.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.299+0000 7fa34b042640 1 -- 172.21.15.69:0/3644316452 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa34409da70 con 0x7fa3440b17c0 2024-04-25T10:59:43.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.299+0000 7fa3417fa640 1 -- 172.21.15.69:0/3644316452 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fa3380802a0 con 0x7fa3440b17c0 2024-04-25T10:59:43.300 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-25T10:59:43.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.301+0000 7fa34b042640 1 -- 172.21.15.69:0/3644316452 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa3300744e0 msgr2=0x7fa3300769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:43.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.301+0000 7fa34b042640 1 --2- 172.21.15.69:0/3644316452 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa3300744e0 0x7fa3300769f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa334006fd0 tx=0x7fa334008040 comp rx=0 tx=0).stop 2024-04-25T10:59:43.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.301+0000 7fa34b042640 1 -- 172.21.15.69:0/3644316452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3440b17c0 msgr2=0x7fa34409cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:43.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.301+0000 7fa34b042640 1 --2- 172.21.15.69:0/3644316452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3440b17c0 0x7fa34409cd30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa338009990 tx=0x7fa3380046c0 comp rx=0 tx=0).stop 2024-04-25T10:59:43.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.301+0000 7fa34b042640 1 -- 172.21.15.69:0/3644316452 shutdown_connections 2024-04-25T10:59:43.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.301+0000 7fa34b042640 1 --2- 172.21.15.69:0/3644316452 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa3300744e0 0x7fa3300769f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:43.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.301+0000 7fa34b042640 1 --2- 172.21.15.69:0/3644316452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa344161a50 0x7fa34415f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:43.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.301+0000 7fa34b042640 1 --2- 172.21.15.69:0/3644316452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3440b17c0 0x7fa34409cd30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:43.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.301+0000 7fa34b042640 1 --2- 172.21.15.69:0/3644316452 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa34406dba0 0x7fa34414d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:43.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.301+0000 7fa34b042640 1 -- 172.21.15.69:0/3644316452 >> 172.21.15.69:0/3644316452 conn(0x7fa34409b7d0 msgr2=0x7fa34415e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:43.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.302+0000 7fa34b042640 1 -- 172.21.15.69:0/3644316452 shutdown_connections 2024-04-25T10:59:43.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.302+0000 7fa34b042640 1 -- 172.21.15.69:0/3644316452 wait complete. 2024-04-25T10:59:43.320 INFO:tasks.cephfs_test_runner:Starting test: test_replay_create (tasks.cephfs.test_cap_flush.TestCapFlush) MDS starts to handle client caps when it enters clientreplay stage. 2024-04-25T10:59:43.320 INFO:tasks.cephfs_test_runner:test_replay_create (tasks.cephfs.test_cap_flush.TestCapFlush) 2024-04-25T10:59:43.321 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:59:43.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.428+0000 7f1f19b84640 1 Processor -- start 2024-04-25T10:59:43.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.428+0000 7f1f19b84640 1 -- start start 2024-04-25T10:59:43.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.429+0000 7f1f19b84640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f140b3f80 0x7f1f140d1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:43.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.429+0000 7f1f19b84640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1f140d2240 0x7f1f140b2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:43.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.429+0000 7f1f19b84640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f140b4390 con 0x7f1f14160870 2024-04-25T10:59:43.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.429+0000 7f1f19b84640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f1f140d00f0 con 0x7f1f140d2240 2024-04-25T10:59:43.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.429+0000 7f1f19b84640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1f140b2470 con 0x7f1f140b3f80 2024-04-25T10:59:43.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.429+0000 7f1f137fe640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f1f14160870 0x7f1f140cf300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:44320/0 (socket says 172.21.15.69:44320) 2024-04-25T10:59:43.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.429+0000 7f1f137fe640 1 -- 172.21.15.69:0/633662387 learned_addr learned my addr 172.21.15.69:0/633662387 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:43.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.430+0000 7f1f13fff640 1 --2- 172.21.15.69:0/633662387 >> v2:172.21.15.146:3300/0 conn(0x7f1f140d2240 0x7f1f140b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:43.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.430+0000 7f1f12ffd640 1 --2- 172.21.15.69:0/633662387 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f140b3f80 0x7f1f140d1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:43.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.430+0000 7f1f127fc640 1 -- 172.21.15.69:0/633662387 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2769044316 0 0) 0x7f1f04001110 con 0x7f1f14160870 2024-04-25T10:59:43.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.430+0000 7f1f127fc640 1 -- 172.21.15.69:0/633662387 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1efc003520 con 0x7f1f14160870 2024-04-25T10:59:43.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.430+0000 7f1f127fc640 1 -- 172.21.15.69:0/633662387 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1878115207 0 0) 0x7f1efc003520 con 0x7f1f14160870 2024-04-25T10:59:43.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.430+0000 7f1f127fc640 1 -- 172.21.15.69:0/633662387 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f140b3f80 msgr2=0x7f1f140d1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:43.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.430+0000 7f1f127fc640 1 --2- 172.21.15.69:0/633662387 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f140b3f80 0x7f1f140d1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:43.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.430+0000 7f1f127fc640 1 -- 172.21.15.69:0/633662387 >> v2:172.21.15.146:3300/0 conn(0x7f1f140d2240 msgr2=0x7f1f140b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:43.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.430+0000 7f1f127fc640 1 --2- 172.21.15.69:0/633662387 >> v2:172.21.15.146:3300/0 conn(0x7f1f140d2240 0x7f1f140b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:43.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.430+0000 7f1f127fc640 1 -- 172.21.15.69:0/633662387 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f140cfb30 con 0x7f1f14160870 2024-04-25T10:59:43.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.430+0000 7f1f127fc640 1 -- 172.21.15.69:0/633662387 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7f1f04001840 con 0x7f1f14160870 2024-04-25T10:59:43.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.430+0000 7f1f13fff640 1 --2- 172.21.15.69:0/633662387 >> v2:172.21.15.146:3300/0 conn(0x7f1f140d2240 0x7f1f140b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:59:43.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.430+0000 7f1f127fc640 1 -- 172.21.15.69:0/633662387 >> v1:172.21.15.69:6789/0 conn(0x7f1f14160870 legacy=0x7f1f140cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:43.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.430+0000 7f1f127fc640 1 --2- 172.21.15.69:0/633662387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1efc0039b0 0x7f1efc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:43.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.430+0000 7f1f127fc640 1 --2- 172.21.15.69:0/633662387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1efc004800 0x7f1efc004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:43.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.430+0000 7f1f127fc640 1 --2- 172.21.15.69:0/633662387 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f140b3f80 0x7f1efc005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:43.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.430+0000 7f1f127fc640 1 -- 172.21.15.69:0/633662387 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1f140d00f0 con 0x7f1efc004800 2024-04-25T10:59:43.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.430+0000 7f1f127fc640 1 -- 172.21.15.69:0/633662387 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1f140b2470 con 0x7f1efc0039b0 2024-04-25T10:59:43.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.430+0000 7f1f127fc640 1 -- 172.21.15.69:0/633662387 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1efc00b9c0 con 0x7f1f140b3f80 2024-04-25T10:59:43.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.430+0000 7f1f12ffd640 1 --2- 172.21.15.69:0/633662387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1efc004800 0x7f1efc004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:43.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.430+0000 7f1f137fe640 1 --2- 172.21.15.69:0/633662387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1efc0039b0 0x7f1efc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:43.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.430+0000 7f1f13fff640 1 --2- 172.21.15.69:0/633662387 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f140b3f80 0x7f1efc005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:43.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.430+0000 7f1f12ffd640 1 -- 172.21.15.69:0/633662387 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f140b3f80 msgr2=0x7f1efc005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:43.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.430+0000 7f1f12ffd640 1 --2- 172.21.15.69:0/633662387 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f140b3f80 0x7f1efc005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:43.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.430+0000 7f1f12ffd640 1 -- 172.21.15.69:0/633662387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1efc0039b0 msgr2=0x7f1efc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:43.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.430+0000 7f1f12ffd640 1 --2- 172.21.15.69:0/633662387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1efc0039b0 0x7f1efc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:43.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.431+0000 7f1f12ffd640 1 -- 172.21.15.69:0/633662387 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1efc003520 con 0x7f1efc004800 2024-04-25T10:59:43.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.431+0000 7f1f12ffd640 1 --2- 172.21.15.69:0/633662387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1efc004800 0x7f1efc004c10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009ac0 tx=0x7f1efc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc8ddfc79aacfee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:43.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.431+0000 7f1f127fc640 1 -- 172.21.15.69:0/633662387 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08036070 con 0x7f1efc004800 2024-04-25T10:59:43.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.431+0000 7f1f127fc640 1 -- 172.21.15.69:0/633662387 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f08031400 con 0x7f1efc004800 2024-04-25T10:59:43.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.431+0000 7f1f127fc640 1 -- 172.21.15.69:0/633662387 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08009690 con 0x7f1efc004800 2024-04-25T10:59:43.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.432+0000 7f1f19b84640 1 -- 172.21.15.69:0/633662387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1efc004800 msgr2=0x7f1efc004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:43.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.432+0000 7f1f19b84640 1 --2- 172.21.15.69:0/633662387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1efc004800 0x7f1efc004c10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009ac0 tx=0x7f1efc005830 comp rx=0 tx=0).stop 2024-04-25T10:59:43.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.432+0000 7f1f13fff640 1 -- 172.21.15.69:0/633662387 reap_dead start 2024-04-25T10:59:43.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.432+0000 7f1f19b84640 1 -- 172.21.15.69:0/633662387 shutdown_connections 2024-04-25T10:59:43.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.432+0000 7f1f19b84640 1 -- 172.21.15.69:0/633662387 >> 172.21.15.69:0/633662387 conn(0x7f1f1409b7d0 msgr2=0x7f1f1409bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:43.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.432+0000 7f1f19b84640 1 -- 172.21.15.69:0/633662387 shutdown_connections 2024-04-25T10:59:43.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.432+0000 7f1f19b84640 1 -- 172.21.15.69:0/633662387 wait complete. 2024-04-25T10:59:43.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.434+0000 7f1f19b84640 1 Processor -- start 2024-04-25T10:59:43.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.434+0000 7f1f19b84640 1 -- start start 2024-04-25T10:59:43.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.435+0000 7f1f19b84640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1efc004800 0x7f1f14152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:43.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.435+0000 7f1f19b84640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f1406e810 0x7f1f140c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:43.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.435+0000 7f1f19b84640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f1406ec20 0x7f1f140c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:43.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.435+0000 7f1f19b84640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1f140d2650 con 0x7f1f1406ec20 2024-04-25T10:59:43.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.435+0000 7f1f19b84640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1f140b2100 con 0x7f1efc004800 2024-04-25T10:59:43.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.435+0000 7f1f19b84640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1f140af850 con 0x7f1f1406e810 2024-04-25T10:59:43.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.435+0000 7f1f13fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f1406ec20 0x7f1f140c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:43.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.435+0000 7f1f13fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f1406ec20 0x7f1f140c8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43860/0 (socket says 172.21.15.69:43860) 2024-04-25T10:59:43.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.435+0000 7f1f13fff640 1 -- 172.21.15.69:0/3250290221 learned_addr learned my addr 172.21.15.69:0/3250290221 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:43.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.435+0000 7f1f12ffd640 1 --2- 172.21.15.69:0/3250290221 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f1406e810 0x7f1f140c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:43.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.435+0000 7f1f137fe640 1 --2- 172.21.15.69:0/3250290221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1efc004800 0x7f1f14152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:43.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.436+0000 7f1f13fff640 1 -- 172.21.15.69:0/3250290221 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f1406e810 msgr2=0x7f1f140c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:43.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.436+0000 7f1f13fff640 1 --2- 172.21.15.69:0/3250290221 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f1406e810 0x7f1f140c8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:43.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.436+0000 7f1f13fff640 1 -- 172.21.15.69:0/3250290221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1efc004800 msgr2=0x7f1f14152ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:43.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.436+0000 7f1f13fff640 1 --2- 172.21.15.69:0/3250290221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1efc004800 0x7f1f14152ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:43.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.436+0000 7f1f13fff640 1 -- 172.21.15.69:0/3250290221 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f140c70c0 con 0x7f1f1406ec20 2024-04-25T10:59:43.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.436+0000 7f1f13fff640 1 --2- 172.21.15.69:0/3250290221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f1406ec20 0x7f1f140c8990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1f1406fc10 tx=0x7f1f0c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:43.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.436+0000 7f1efbfff640 1 -- 172.21.15.69:0/3250290221 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c00fdc0 con 0x7f1f1406ec20 2024-04-25T10:59:43.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.436+0000 7f1efbfff640 1 -- 172.21.15.69:0/3250290221 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f0c003990 con 0x7f1f1406ec20 2024-04-25T10:59:43.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.436+0000 7f1efbfff640 1 -- 172.21.15.69:0/3250290221 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c003d80 con 0x7f1f1406ec20 2024-04-25T10:59:43.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.436+0000 7f1f19b84640 1 -- 172.21.15.69:0/3250290221 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f140c73a0 con 0x7f1f1406ec20 2024-04-25T10:59:43.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.436+0000 7f1f19b84640 1 -- 172.21.15.69:0/3250290221 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f140c7960 con 0x7f1f1406ec20 2024-04-25T10:59:43.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.437+0000 7f1f19b84640 1 -- 172.21.15.69:0/3250290221 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f14050740 con 0x7f1f1406ec20 2024-04-25T10:59:43.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.437+0000 7f1efbfff640 1 -- 172.21.15.69:0/3250290221 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1f0c01f020 con 0x7f1f1406ec20 2024-04-25T10:59:43.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.438+0000 7f1efbfff640 1 --2- 172.21.15.69:0/3250290221 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1f000744e0 0x7f1f000769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:43.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.438+0000 7f1f137fe640 1 --2- 172.21.15.69:0/3250290221 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1f000744e0 0x7f1f000769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:43.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.438+0000 7f1efbfff640 1 -- 172.21.15.69:0/3250290221 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1f0c095740 con 0x7f1f1406ec20 2024-04-25T10:59:43.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.438+0000 7f1f137fe640 1 --2- 172.21.15.69:0/3250290221 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1f000744e0 0x7f1f000769f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1f040012f0 tx=0x7f1f04009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:43.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.442+0000 7f1efbfff640 1 -- 172.21.15.69:0/3250290221 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1f0c010040 con 0x7f1f1406ec20 2024-04-25T10:59:43.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.617+0000 7f1f19b84640 1 -- 172.21.15.69:0/3250290221 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f140b0380 con 0x7f1f1406ec20 2024-04-25T10:59:43.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.619+0000 7f1efbfff640 1 -- 172.21.15.69:0/3250290221 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f1f0c05fed0 con 0x7f1f1406ec20 2024-04-25T10:59:43.711 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:43.711 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":20,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T10:59:20.630151+0000","last_up_change":"2024-04-25T10:59:05.546839+0000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:59:06.422674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:59:16.727980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:59:17.938486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6825","nonce":1876101065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6827","nonce":1876101065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6831","nonce":1876101065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6829","nonce":1876101065}]},"public_addr":"172.21.15.69:6825/1876101065","cluster_addr":"172.21.15.69:6827/1876101065","heartbeat_back_addr":"172.21.15.69:6831/1876101065","heartbeat_front_addr":"172.21.15.69:6829/1876101065","state":["exists","up"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6801","nonce":1806438563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6803","nonce":1806438563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6807","nonce":1806438563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6805","nonce":1806438563}]},"public_addr":"172.21.15.69:6801/1806438563","cluster_addr":"172.21.15.69:6803/1806438563","heartbeat_back_addr":"172.21.15.69:6807/1806438563","heartbeat_front_addr":"172.21.15.69:6805/1806438563","state":["exists","up"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6817","nonce":1593617983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6819","nonce":1593617983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6823","nonce":1593617983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6821","nonce":1593617983}]},"public_addr":"172.21.15.69:6817/1593617983","cluster_addr":"172.21.15.69:6819/1593617983","heartbeat_back_addr":"172.21.15.69:6823/1593617983","heartbeat_front_addr":"172.21.15.69:6821/1593617983","state":["exists","up"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6809","nonce":3950661294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6811","nonce":3950661294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6815","nonce":3950661294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6813","nonce":3950661294}]},"public_addr":"172.21.15.69:6809/3950661294","cluster_addr":"172.21.15.69:6811/3950661294","heartbeat_back_addr":"172.21.15.69:6815/3950661294","heartbeat_front_addr":"172.21.15.69:6813/3950661294","state":["exists","up"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6801","nonce":2508929872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6803","nonce":2508929872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6807","nonce":2508929872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6805","nonce":2508929872}]},"public_addr":"172.21.15.146:6801/2508929872","cluster_addr":"172.21.15.146:6803/2508929872","heartbeat_back_addr":"172.21.15.146:6807/2508929872","heartbeat_front_addr":"172.21.15.146:6805/2508929872","state":["exists","up"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6811","nonce":1310911138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6816","nonce":1310911138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6823","nonce":1310911138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6821","nonce":1310911138}]},"public_addr":"172.21.15.146:6811/1310911138","cluster_addr":"172.21.15.146:6816/1310911138","heartbeat_back_addr":"172.21.15.146:6823/1310911138","heartbeat_front_addr":"172.21.15.146:6821/1310911138","state":["exists","up"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6810","nonce":634688021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6813","nonce":634688021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6820","nonce":634688021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6817","nonce":634688021}]},"public_addr":"172.21.15.146:6810/634688021","cluster_addr":"172.21.15.146:6813/634688021","heartbeat_back_addr":"172.21.15.146:6820/634688021","heartbeat_front_addr":"172.21.15.146:6817/634688021","state":["exists","up"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6825","nonce":3814215935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6827","nonce":3814215935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6831","nonce":3814215935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6829","nonce":3814215935}]},"public_addr":"172.21.15.146:6825/3814215935","cluster_addr":"172.21.15.146:6827/3814215935","heartbeat_back_addr":"172.21.15.146:6831/3814215935","heartbeat_front_addr":"172.21.15.146:6829/3814215935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.820200+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.850189+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.764560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.866248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.807897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.869881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.782310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.827131+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:59:43.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.621+0000 7f1f19b84640 1 -- 172.21.15.69:0/3250290221 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1f000744e0 msgr2=0x7f1f000769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:43.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.621+0000 7f1f19b84640 1 --2- 172.21.15.69:0/3250290221 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1f000744e0 0x7f1f000769f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1f040012f0 tx=0x7f1f04009040 comp rx=0 tx=0).stop 2024-04-25T10:59:43.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.621+0000 7f1f19b84640 1 -- 172.21.15.69:0/3250290221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f1406ec20 msgr2=0x7f1f140c8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:43.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.621+0000 7f1f19b84640 1 --2- 172.21.15.69:0/3250290221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f1406ec20 0x7f1f140c8990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1f1406fc10 tx=0x7f1f0c00f580 comp rx=0 tx=0).stop 2024-04-25T10:59:43.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.621+0000 7f1f19b84640 1 -- 172.21.15.69:0/3250290221 shutdown_connections 2024-04-25T10:59:43.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.621+0000 7f1f19b84640 1 --2- 172.21.15.69:0/3250290221 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1f000744e0 0x7f1f000769f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:43.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.621+0000 7f1f19b84640 1 --2- 172.21.15.69:0/3250290221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1f1406ec20 0x7f1f140c8990 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:43.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.621+0000 7f1f19b84640 1 --2- 172.21.15.69:0/3250290221 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1f1406e810 0x7f1f140c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:43.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.621+0000 7f1f19b84640 1 --2- 172.21.15.69:0/3250290221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1efc004800 0x7f1f14152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:43.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.621+0000 7f1f19b84640 1 -- 172.21.15.69:0/3250290221 >> 172.21.15.69:0/3250290221 conn(0x7f1f1409b7d0 msgr2=0x7f1f140b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:43.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.621+0000 7f1f19b84640 1 -- 172.21.15.69:0/3250290221 shutdown_connections 2024-04-25T10:59:43.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.621+0000 7f1f19b84640 1 -- 172.21.15.69:0/3250290221 wait complete. 2024-04-25T10:59:43.713 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:59:06.422674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:59:16.727980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:59:17.938486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:59:43.714 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:59:43.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.826+0000 7fd01936c640 1 Processor -- start 2024-04-25T10:59:43.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.826+0000 7fd01936c640 1 -- start start 2024-04-25T10:59:43.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.827+0000 7fd01936c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fd0140b1c50 0x7fd0140b90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:43.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.827+0000 7fd01936c640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0140d21e0 con 0x7fd01408dec0 2024-04-25T10:59:43.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.827+0000 7fd01936c640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0140d1940 con 0x7fd01408d630 2024-04-25T10:59:43.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.827+0000 7fd01936c640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fd0140b95f0 con 0x7fd0140b1c50 2024-04-25T10:59:43.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.827+0000 7fd0137fe640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fd0140b1c50 0x7fd0140b90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:43.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.827+0000 7fd012ffd640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fd01408dec0 0x7fd01408cf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:44326/0 (socket says 172.21.15.69:44326) 2024-04-25T10:59:43.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.827+0000 7fd012ffd640 1 -- 172.21.15.69:0/1535942257 learned_addr learned my addr 172.21.15.69:0/1535942257 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:43.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.828+0000 7fd0137fe640 1 -- 172.21.15.69:0/1535942257 >> v1:172.21.15.146:6789/0 conn(0x7fd01408d630 legacy=0x7fd0140ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:43.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.828+0000 7fd0137fe640 1 -- 172.21.15.69:0/1535942257 >> v1:172.21.15.69:6789/0 conn(0x7fd01408dec0 legacy=0x7fd01408cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:43.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.828+0000 7fd0127fc640 1 -- 172.21.15.69:0/1535942257 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 244954733 0 0) 0x7fd0140d21e0 con 0x7fd01408dec0 2024-04-25T10:59:43.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.828+0000 7fd0137fe640 1 -- 172.21.15.69:0/1535942257 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0140d1c20 con 0x7fd0140b1c50 2024-04-25T10:59:43.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.828+0000 7fd0137fe640 1 --2- 172.21.15.69:0/1535942257 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0140b1c50 0x7fd0140b90b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd00c009d90 tx=0x7fd00c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39de626e06a77c74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:43.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.829+0000 7fd0127fc640 1 -- 172.21.15.69:0/1535942257 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c007740 con 0x7fd0140b1c50 2024-04-25T10:59:43.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.829+0000 7fd0127fc640 1 -- 172.21.15.69:0/1535942257 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd00c0078e0 con 0x7fd0140b1c50 2024-04-25T10:59:43.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.829+0000 7fd0127fc640 1 -- 172.21.15.69:0/1535942257 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c007cb0 con 0x7fd0140b1c50 2024-04-25T10:59:43.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.829+0000 7fd01936c640 1 -- 172.21.15.69:0/1535942257 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0140b1c50 msgr2=0x7fd0140b90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:43.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.829+0000 7fd01936c640 1 --2- 172.21.15.69:0/1535942257 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0140b1c50 0x7fd0140b90b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd00c009d90 tx=0x7fd00c02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:59:43.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.829+0000 7fd01936c640 1 -- 172.21.15.69:0/1535942257 shutdown_connections 2024-04-25T10:59:43.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.829+0000 7fd01936c640 1 --2- 172.21.15.69:0/1535942257 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0140b1c50 0x7fd0140b90b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:43.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.829+0000 7fd01936c640 1 -- 172.21.15.69:0/1535942257 >> 172.21.15.69:0/1535942257 conn(0x7fd0140d0510 msgr2=0x7fd0140d0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:43.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.829+0000 7fd01936c640 1 -- 172.21.15.69:0/1535942257 shutdown_connections 2024-04-25T10:59:43.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.829+0000 7fd01936c640 1 -- 172.21.15.69:0/1535942257 wait complete. 2024-04-25T10:59:43.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.831+0000 7fd01936c640 1 Processor -- start 2024-04-25T10:59:43.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.831+0000 7fd01936c640 1 -- start start 2024-04-25T10:59:43.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.832+0000 7fd01936c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd01408dec0 0x7fd0141e95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:43.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.832+0000 7fd01936c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0140b1c50 0x7fd0141512c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:43.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.832+0000 7fd01936c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd01415a4a0 0x7fd01415a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:43.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.832+0000 7fd01936c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd0140d15d0 con 0x7fd0140b1c50 2024-04-25T10:59:43.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.832+0000 7fd01936c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd0140b0720 con 0x7fd01408dec0 2024-04-25T10:59:43.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.832+0000 7fd01936c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd0140cef90 con 0x7fd01415a4a0 2024-04-25T10:59:43.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.833+0000 7fd00bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0140b1c50 0x7fd0141512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:43.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.833+0000 7fd00bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0140b1c50 0x7fd0141512c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43872/0 (socket says 172.21.15.69:43872) 2024-04-25T10:59:43.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.833+0000 7fd00bfff640 1 -- 172.21.15.69:0/2995690346 learned_addr learned my addr 172.21.15.69:0/2995690346 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:43.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.833+0000 7fd012ffd640 1 --2- 172.21.15.69:0/2995690346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd01408dec0 0x7fd0141e95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:43.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.833+0000 7fd0137fe640 1 --2- 172.21.15.69:0/2995690346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd01415a4a0 0x7fd01415a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:43.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.833+0000 7fd00bfff640 1 -- 172.21.15.69:0/2995690346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd01415a4a0 msgr2=0x7fd01415a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:43.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.833+0000 7fd00bfff640 1 --2- 172.21.15.69:0/2995690346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd01415a4a0 0x7fd01415a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:43.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.833+0000 7fd00bfff640 1 -- 172.21.15.69:0/2995690346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd01408dec0 msgr2=0x7fd0141e95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:43.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.833+0000 7fd00bfff640 1 --2- 172.21.15.69:0/2995690346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd01408dec0 0x7fd0141e95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:43.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.833+0000 7fd00bfff640 1 -- 172.21.15.69:0/2995690346 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd01415adf0 con 0x7fd0140b1c50 2024-04-25T10:59:43.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.833+0000 7fd0137fe640 1 --2- 172.21.15.69:0/2995690346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd01415a4a0 0x7fd01415a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:43.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.833+0000 7fd00bfff640 1 --2- 172.21.15.69:0/2995690346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0140b1c50 0x7fd0141512c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcffc00b3f0 tx=0x7fcffc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:43.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.833+0000 7fd00b7fe640 1 -- 172.21.15.69:0/2995690346 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcffc015020 con 0x7fd0140b1c50 2024-04-25T10:59:43.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.833+0000 7fd00b7fe640 1 -- 172.21.15.69:0/2995690346 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcffc011830 con 0x7fd0140b1c50 2024-04-25T10:59:43.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.833+0000 7fd01936c640 1 -- 172.21.15.69:0/2995690346 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0140caf60 con 0x7fd0140b1c50 2024-04-25T10:59:43.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.833+0000 7fd00b7fe640 1 -- 172.21.15.69:0/2995690346 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcffc011c20 con 0x7fd0140b1c50 2024-04-25T10:59:43.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.833+0000 7fd01936c640 1 -- 172.21.15.69:0/2995690346 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0140cb520 con 0x7fd0140b1c50 2024-04-25T10:59:43.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.834+0000 7fd01936c640 1 -- 172.21.15.69:0/2995690346 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd014050740 con 0x7fd0140b1c50 2024-04-25T10:59:43.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.835+0000 7fd00b7fe640 1 -- 172.21.15.69:0/2995690346 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcffc018020 con 0x7fd0140b1c50 2024-04-25T10:59:43.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.835+0000 7fd00b7fe640 1 --2- 172.21.15.69:0/2995690346 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fcff40744b0 0x7fcff40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:43.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.835+0000 7fd012ffd640 1 --2- 172.21.15.69:0/2995690346 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fcff40744b0 0x7fcff40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:43.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.835+0000 7fd00b7fe640 1 -- 172.21.15.69:0/2995690346 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcffc094e80 con 0x7fd0140b1c50 2024-04-25T10:59:43.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.836+0000 7fd012ffd640 1 --2- 172.21.15.69:0/2995690346 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fcff40744b0 0x7fcff40769c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd004000f60 tx=0x7fd004007c00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:43.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:43.840+0000 7fd00b7fe640 1 -- 172.21.15.69:0/2995690346 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcffc01f030 con 0x7fd0140b1c50 2024-04-25T10:59:44.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.014+0000 7fd01936c640 1 -- 172.21.15.69:0/2995690346 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd0140cacf0 con 0x7fd0140b1c50 2024-04-25T10:59:44.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.014+0000 7fd00b7fe640 1 -- 172.21.15.69:0/2995690346 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fd01415adf0 con 0x7fd0140b1c50 2024-04-25T10:59:44.015 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-25T10:59:44.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.016+0000 7fd01936c640 1 -- 172.21.15.69:0/2995690346 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fcff40744b0 msgr2=0x7fcff40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:44.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.016+0000 7fd01936c640 1 --2- 172.21.15.69:0/2995690346 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fcff40744b0 0x7fcff40769c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd004000f60 tx=0x7fd004007c00 comp rx=0 tx=0).stop 2024-04-25T10:59:44.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.016+0000 7fd01936c640 1 -- 172.21.15.69:0/2995690346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0140b1c50 msgr2=0x7fd0141512c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:44.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.016+0000 7fd01936c640 1 --2- 172.21.15.69:0/2995690346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0140b1c50 0x7fd0141512c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcffc00b3f0 tx=0x7fcffc00b8b0 comp rx=0 tx=0).stop 2024-04-25T10:59:44.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.016+0000 7fd01936c640 1 -- 172.21.15.69:0/2995690346 shutdown_connections 2024-04-25T10:59:44.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.016+0000 7fd01936c640 1 --2- 172.21.15.69:0/2995690346 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fcff40744b0 0x7fcff40769c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:44.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.016+0000 7fd01936c640 1 --2- 172.21.15.69:0/2995690346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd01415a4a0 0x7fd01415a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:44.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.016+0000 7fd01936c640 1 --2- 172.21.15.69:0/2995690346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0140b1c50 0x7fd0141512c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:44.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.016+0000 7fd01936c640 1 --2- 172.21.15.69:0/2995690346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd01408dec0 0x7fd0141e95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:44.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.016+0000 7fd01936c640 1 -- 172.21.15.69:0/2995690346 >> 172.21.15.69:0/2995690346 conn(0x7fd0140d0510 msgr2=0x7fd01415df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:44.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.016+0000 7fd01936c640 1 -- 172.21.15.69:0/2995690346 shutdown_connections 2024-04-25T10:59:44.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.016+0000 7fd01936c640 1 -- 172.21.15.69:0/2995690346 wait complete. 2024-04-25T10:59:44.028 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:59:44.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.137+0000 7fd631134640 1 Processor -- start 2024-04-25T10:59:44.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.137+0000 7fd631134640 1 -- start start 2024-04-25T10:59:44.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.139+0000 7fd631134640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd62c0b9270 0x7fd62c0d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:44.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.139+0000 7fd631134640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd62c161ef0 0x7fd62c162300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:44.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.139+0000 7fd631134640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd62c0b0690 con 0x7fd62c162840 2024-04-25T10:59:44.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.139+0000 7fd631134640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fd62c0cfab0 con 0x7fd62c161ef0 2024-04-25T10:59:44.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.139+0000 7fd631134640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd62c0cfd90 con 0x7fd62c0b9270 2024-04-25T10:59:44.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.139+0000 7fd62a59c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd62c161ef0 0x7fd62c162300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:44.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.139+0000 7fd62a59c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd62c161ef0 0x7fd62c162300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.69:49818/0 (socket says 172.21.15.69:49818) 2024-04-25T10:59:44.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.139+0000 7fd62a59c640 1 -- 172.21.15.69:0/1098914011 learned_addr learned my addr 172.21.15.69:0/1098914011 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:44.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.139+0000 7fd62ad9d640 1 --2- 172.21.15.69:0/1098914011 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd62c0b9270 0x7fd62c0d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:44.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.139+0000 7fd62a59c640 1 -- 172.21.15.69:0/1098914011 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd62c0b9270 msgr2=0x7fd62c0d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:44.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.139+0000 7fd62a59c640 1 --2- 172.21.15.69:0/1098914011 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd62c0b9270 0x7fd62c0d15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:44.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.139+0000 7fd62a59c640 1 -- 172.21.15.69:0/1098914011 >> v1:172.21.15.146:6789/0 conn(0x7fd62c162840 legacy=0x7fd62c0ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:44.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.139+0000 7fd62a59c640 1 -- 172.21.15.69:0/1098914011 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd62c162c50 con 0x7fd62c161ef0 2024-04-25T10:59:44.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.140+0000 7fd62a59c640 1 --2- 172.21.15.69:0/1098914011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd62c161ef0 0x7fd62c162300 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd620009870 tx=0x7fd62002a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=522f48d073ef18ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:44.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.140+0000 7fd629d9b640 1 -- 172.21.15.69:0/1098914011 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd620037070 con 0x7fd62c161ef0 2024-04-25T10:59:44.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.140+0000 7fd629d9b640 1 -- 172.21.15.69:0/1098914011 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd620032440 con 0x7fd62c161ef0 2024-04-25T10:59:44.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.140+0000 7fd629d9b640 1 -- 172.21.15.69:0/1098914011 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd620032810 con 0x7fd62c161ef0 2024-04-25T10:59:44.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.141+0000 7fd631134640 1 -- 172.21.15.69:0/1098914011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd62c161ef0 msgr2=0x7fd62c162300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:44.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.141+0000 7fd631134640 1 --2- 172.21.15.69:0/1098914011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd62c161ef0 0x7fd62c162300 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd620009870 tx=0x7fd62002a130 comp rx=0 tx=0).stop 2024-04-25T10:59:44.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.141+0000 7fd631134640 1 -- 172.21.15.69:0/1098914011 shutdown_connections 2024-04-25T10:59:44.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.141+0000 7fd631134640 1 --2- 172.21.15.69:0/1098914011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd62c161ef0 0x7fd62c162300 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:44.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.141+0000 7fd631134640 1 --2- 172.21.15.69:0/1098914011 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd62c0b9270 0x7fd62c0d15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:44.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.141+0000 7fd631134640 1 -- 172.21.15.69:0/1098914011 >> 172.21.15.69:0/1098914011 conn(0x7fd62c0d3aa0 msgr2=0x7fd62c0d3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:44.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.141+0000 7fd631134640 1 -- 172.21.15.69:0/1098914011 shutdown_connections 2024-04-25T10:59:44.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.141+0000 7fd631134640 1 -- 172.21.15.69:0/1098914011 wait complete. 2024-04-25T10:59:44.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.143+0000 7fd631134640 1 Processor -- start 2024-04-25T10:59:44.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.143+0000 7fd631134640 1 -- start start 2024-04-25T10:59:44.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.144+0000 7fd631134640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd62c0b9270 0x7fd62c06e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:44.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.144+0000 7fd631134640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd62c161ef0 0x7fd62c1f9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:44.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.145+0000 7fd631134640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd62c162840 0x7fd62c0cc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:44.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.145+0000 7fd631134640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd62c0cef10 con 0x7fd62c161ef0 2024-04-25T10:59:44.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.145+0000 7fd631134640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd62c0b2110 con 0x7fd62c0b9270 2024-04-25T10:59:44.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.145+0000 7fd631134640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd62c0cf090 con 0x7fd62c162840 2024-04-25T10:59:44.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.145+0000 7fd62a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd62c161ef0 0x7fd62c1f9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:44.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.145+0000 7fd62a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd62c161ef0 0x7fd62c1f9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43886/0 (socket says 172.21.15.69:43886) 2024-04-25T10:59:44.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.145+0000 7fd62a59c640 1 -- 172.21.15.69:0/2221856912 learned_addr learned my addr 172.21.15.69:0/2221856912 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:44.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.145+0000 7fd62ad9d640 1 --2- 172.21.15.69:0/2221856912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd62c0b9270 0x7fd62c06e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:44.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.145+0000 7fd62b59e640 1 --2- 172.21.15.69:0/2221856912 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd62c162840 0x7fd62c0cc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:44.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.145+0000 7fd62a59c640 1 -- 172.21.15.69:0/2221856912 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd62c162840 msgr2=0x7fd62c0cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:44.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.145+0000 7fd62a59c640 1 --2- 172.21.15.69:0/2221856912 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd62c162840 0x7fd62c0cc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:44.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.145+0000 7fd62a59c640 1 -- 172.21.15.69:0/2221856912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd62c0b9270 msgr2=0x7fd62c06e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:44.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.145+0000 7fd62a59c640 1 --2- 172.21.15.69:0/2221856912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd62c0b9270 0x7fd62c06e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:44.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.145+0000 7fd62a59c640 1 -- 172.21.15.69:0/2221856912 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd62c0b2ba0 con 0x7fd62c161ef0 2024-04-25T10:59:44.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.145+0000 7fd62a59c640 1 --2- 172.21.15.69:0/2221856912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd62c161ef0 0x7fd62c1f9830 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd62002a170 tx=0x7fd6200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:44.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.145+0000 7fd6137fe640 1 -- 172.21.15.69:0/2221856912 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd620037040 con 0x7fd62c161ef0 2024-04-25T10:59:44.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.145+0000 7fd6137fe640 1 -- 172.21.15.69:0/2221856912 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd620042070 con 0x7fd62c161ef0 2024-04-25T10:59:44.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.145+0000 7fd631134640 1 -- 172.21.15.69:0/2221856912 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd62c0b2e80 con 0x7fd62c161ef0 2024-04-25T10:59:44.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.145+0000 7fd6137fe640 1 -- 172.21.15.69:0/2221856912 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd620030880 con 0x7fd62c161ef0 2024-04-25T10:59:44.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.145+0000 7fd631134640 1 -- 172.21.15.69:0/2221856912 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd62c0b3440 con 0x7fd62c161ef0 2024-04-25T10:59:44.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.147+0000 7fd631134640 1 -- 172.21.15.69:0/2221856912 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd62c0d3470 con 0x7fd62c161ef0 2024-04-25T10:59:44.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.147+0000 7fd6137fe640 1 -- 172.21.15.69:0/2221856912 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd620030a20 con 0x7fd62c161ef0 2024-04-25T10:59:44.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.148+0000 7fd6137fe640 1 --2- 172.21.15.69:0/2221856912 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd6180744e0 0x7fd6180769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:44.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.148+0000 7fd62ad9d640 1 --2- 172.21.15.69:0/2221856912 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd6180744e0 0x7fd6180769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:44.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.148+0000 7fd6137fe640 1 -- 172.21.15.69:0/2221856912 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd6200b59b0 con 0x7fd62c161ef0 2024-04-25T10:59:44.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.148+0000 7fd62ad9d640 1 --2- 172.21.15.69:0/2221856912 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd6180744e0 0x7fd6180769f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd61c011340 tx=0x7fd61c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:44.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.153+0000 7fd6137fe640 1 -- 172.21.15.69:0/2221856912 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd62c0d3470 con 0x7fd62c161ef0 2024-04-25T10:59:44.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.328+0000 7fd631134640 1 -- 172.21.15.69:0/2221856912 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd62c050740 con 0x7fd62c161ef0 2024-04-25T10:59:44.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.328+0000 7fd6137fe640 1 -- 172.21.15.69:0/2221856912 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fd62007faf0 con 0x7fd62c161ef0 2024-04-25T10:59:44.329 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-25T10:59:44.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.330+0000 7fd631134640 1 -- 172.21.15.69:0/2221856912 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd6180744e0 msgr2=0x7fd6180769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:44.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.330+0000 7fd631134640 1 --2- 172.21.15.69:0/2221856912 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd6180744e0 0x7fd6180769f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd61c011340 tx=0x7fd61c00b040 comp rx=0 tx=0).stop 2024-04-25T10:59:44.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.330+0000 7fd631134640 1 -- 172.21.15.69:0/2221856912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd62c161ef0 msgr2=0x7fd62c1f9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:44.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.330+0000 7fd631134640 1 --2- 172.21.15.69:0/2221856912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd62c161ef0 0x7fd62c1f9830 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd62002a170 tx=0x7fd6200046c0 comp rx=0 tx=0).stop 2024-04-25T10:59:44.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.330+0000 7fd631134640 1 -- 172.21.15.69:0/2221856912 shutdown_connections 2024-04-25T10:59:44.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.330+0000 7fd631134640 1 --2- 172.21.15.69:0/2221856912 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd6180744e0 0x7fd6180769f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:44.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.330+0000 7fd631134640 1 --2- 172.21.15.69:0/2221856912 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd62c162840 0x7fd62c0cc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:44.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.330+0000 7fd631134640 1 --2- 172.21.15.69:0/2221856912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd62c161ef0 0x7fd62c1f9830 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:44.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.330+0000 7fd631134640 1 --2- 172.21.15.69:0/2221856912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd62c0b9270 0x7fd62c06e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:44.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.330+0000 7fd631134640 1 -- 172.21.15.69:0/2221856912 >> 172.21.15.69:0/2221856912 conn(0x7fd62c0d3aa0 msgr2=0x7fd62c0d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:44.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.330+0000 7fd631134640 1 -- 172.21.15.69:0/2221856912 shutdown_connections 2024-04-25T10:59:44.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.330+0000 7fd631134640 1 -- 172.21.15.69:0/2221856912 wait complete. 2024-04-25T10:59:44.342 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:59:44.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.452+0000 7fd486020640 1 Processor -- start 2024-04-25T10:59:44.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.452+0000 7fd486020640 1 -- start start 2024-04-25T10:59:44.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.453+0000 7fd486020640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd480164ad0 0x7fd4800b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:44.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.453+0000 7fd486020640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd480163f30 0x7fd480164340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:44.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.453+0000 7fd486020640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd480164880 con 0x7fd4800d1f30 2024-04-25T10:59:44.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.453+0000 7fd486020640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fd480164ee0 con 0x7fd480164ad0 2024-04-25T10:59:44.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.453+0000 7fd486020640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd4800b31e0 con 0x7fd480163f30 2024-04-25T10:59:44.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.453+0000 7fd47ffff640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fd4800d1f30 0x7fd4800d2340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:44330/0 (socket says 172.21.15.69:44330) 2024-04-25T10:59:44.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.454+0000 7fd47ffff640 1 -- 172.21.15.69:0/2485256163 learned_addr learned my addr 172.21.15.69:0/2485256163 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:44.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.454+0000 7fd47f7fe640 1 --2- 172.21.15.69:0/2485256163 >> v2:172.21.15.146:3300/0 conn(0x7fd480164ad0 0x7fd4800b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:44.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.454+0000 7fd47effd640 1 --2- 172.21.15.69:0/2485256163 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd480163f30 0x7fd480164340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:44.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.454+0000 7fd47e7fc640 1 -- 172.21.15.69:0/2485256163 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3682086641 0 0) 0x7fd478001490 con 0x7fd4800d1f30 2024-04-25T10:59:44.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.454+0000 7fd47e7fc640 1 -- 172.21.15.69:0/2485256163 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd468003520 con 0x7fd4800d1f30 2024-04-25T10:59:44.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.454+0000 7fd47e7fc640 1 -- 172.21.15.69:0/2485256163 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1587628029 0 0) 0x7fd468003520 con 0x7fd4800d1f30 2024-04-25T10:59:44.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.454+0000 7fd47f7fe640 1 -- 172.21.15.69:0/2485256163 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd480163f30 msgr2=0x7fd480164340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:44.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.454+0000 7fd47f7fe640 1 --2- 172.21.15.69:0/2485256163 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd480163f30 0x7fd480164340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:44.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.454+0000 7fd47f7fe640 1 -- 172.21.15.69:0/2485256163 >> v1:172.21.15.69:6789/0 conn(0x7fd4800d1f30 legacy=0x7fd4800d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:44.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.454+0000 7fd47f7fe640 1 -- 172.21.15.69:0/2485256163 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4800d3170 con 0x7fd480164ad0 2024-04-25T10:59:44.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.455+0000 7fd47f7fe640 1 --2- 172.21.15.69:0/2485256163 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd480164ad0 0x7fd4800b2ca0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd4700097a0 tx=0x7fd47002a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1c80ee8c3784de7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:44.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.455+0000 7fd47e7fc640 1 -- 172.21.15.69:0/2485256163 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd470037070 con 0x7fd480164ad0 2024-04-25T10:59:44.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.455+0000 7fd47e7fc640 1 -- 172.21.15.69:0/2485256163 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd47002aba0 con 0x7fd480164ad0 2024-04-25T10:59:44.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.455+0000 7fd47e7fc640 1 -- 172.21.15.69:0/2485256163 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478001bc0 con 0x7fd480164ad0 2024-04-25T10:59:44.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.456+0000 7fd486020640 1 -- 172.21.15.69:0/2485256163 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd480164ad0 msgr2=0x7fd4800b2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:44.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.456+0000 7fd486020640 1 --2- 172.21.15.69:0/2485256163 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd480164ad0 0x7fd4800b2ca0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd4700097a0 tx=0x7fd47002a110 comp rx=0 tx=0).stop 2024-04-25T10:59:44.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.456+0000 7fd486020640 1 -- 172.21.15.69:0/2485256163 shutdown_connections 2024-04-25T10:59:44.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.456+0000 7fd486020640 1 --2- 172.21.15.69:0/2485256163 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd480163f30 0x7fd480164340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:44.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.456+0000 7fd486020640 1 --2- 172.21.15.69:0/2485256163 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd480164ad0 0x7fd4800b2ca0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:44.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.456+0000 7fd486020640 1 -- 172.21.15.69:0/2485256163 >> 172.21.15.69:0/2485256163 conn(0x7fd4800d3aa0 msgr2=0x7fd4800d3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:44.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.456+0000 7fd486020640 1 -- 172.21.15.69:0/2485256163 shutdown_connections 2024-04-25T10:59:44.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.456+0000 7fd486020640 1 -- 172.21.15.69:0/2485256163 wait complete. 2024-04-25T10:59:44.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.457+0000 7fd486020640 1 Processor -- start 2024-04-25T10:59:44.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.458+0000 7fd486020640 1 -- start start 2024-04-25T10:59:44.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.459+0000 7fd486020640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4800d1f30 0x7fd48015b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:44.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.459+0000 7fd486020640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd480163f30 0x7fd48015b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:44.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.459+0000 7fd486020640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd480164ad0 0x7fd48015bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:44.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.459+0000 7fd486020640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd48006d9b0 con 0x7fd480163f30 2024-04-25T10:59:44.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.459+0000 7fd486020640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd48006d6d0 con 0x7fd4800d1f30 2024-04-25T10:59:44.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.459+0000 7fd486020640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd4800b9430 con 0x7fd480164ad0 2024-04-25T10:59:44.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.459+0000 7fd47effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd480163f30 0x7fd48015b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:44.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.459+0000 7fd47effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd480163f30 0x7fd48015b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43900/0 (socket says 172.21.15.69:43900) 2024-04-25T10:59:44.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.459+0000 7fd47effd640 1 -- 172.21.15.69:0/2726437456 learned_addr learned my addr 172.21.15.69:0/2726437456 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:44.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.459+0000 7fd47ffff640 1 --2- 172.21.15.69:0/2726437456 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd480164ad0 0x7fd48015bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:44.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.459+0000 7fd47f7fe640 1 --2- 172.21.15.69:0/2726437456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4800d1f30 0x7fd48015b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:44.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.459+0000 7fd47effd640 1 -- 172.21.15.69:0/2726437456 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd480164ad0 msgr2=0x7fd48015bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:44.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.459+0000 7fd47effd640 1 --2- 172.21.15.69:0/2726437456 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd480164ad0 0x7fd48015bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:44.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.459+0000 7fd47effd640 1 -- 172.21.15.69:0/2726437456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4800d1f30 msgr2=0x7fd48015b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:44.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.459+0000 7fd47effd640 1 --2- 172.21.15.69:0/2726437456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4800d1f30 0x7fd48015b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:44.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.459+0000 7fd47effd640 1 -- 172.21.15.69:0/2726437456 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd480156a90 con 0x7fd480163f30 2024-04-25T10:59:44.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.459+0000 7fd47effd640 1 --2- 172.21.15.69:0/2726437456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd480163f30 0x7fd48015b860 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd47400e870 tx=0x7fd47400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:44.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.460+0000 7fd467fff640 1 -- 172.21.15.69:0/2726437456 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd474017070 con 0x7fd480163f30 2024-04-25T10:59:44.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.460+0000 7fd467fff640 1 -- 172.21.15.69:0/2726437456 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd47400c940 con 0x7fd480163f30 2024-04-25T10:59:44.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.460+0000 7fd467fff640 1 -- 172.21.15.69:0/2726437456 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47400cd30 con 0x7fd480163f30 2024-04-25T10:59:44.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.460+0000 7fd486020640 1 -- 172.21.15.69:0/2726437456 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4800b3400 con 0x7fd480163f30 2024-04-25T10:59:44.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.460+0000 7fd486020640 1 -- 172.21.15.69:0/2726437456 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd48015a840 con 0x7fd480163f30 2024-04-25T10:59:44.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.461+0000 7fd486020640 1 -- 172.21.15.69:0/2726437456 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd480072b70 con 0x7fd480163f30 2024-04-25T10:59:44.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.461+0000 7fd467fff640 1 -- 172.21.15.69:0/2726437456 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd47401a070 con 0x7fd480163f30 2024-04-25T10:59:44.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.462+0000 7fd467fff640 1 --2- 172.21.15.69:0/2726437456 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd46c0744b0 0x7fd46c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:44.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.462+0000 7fd467fff640 1 -- 172.21.15.69:0/2726437456 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd4740950f0 con 0x7fd480163f30 2024-04-25T10:59:44.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.462+0000 7fd47f7fe640 1 --2- 172.21.15.69:0/2726437456 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd46c0744b0 0x7fd46c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:44.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.462+0000 7fd47f7fe640 1 --2- 172.21.15.69:0/2726437456 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd46c0744b0 0x7fd46c0769c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd470034000 tx=0x7fd47002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:44.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.466+0000 7fd467fff640 1 -- 172.21.15.69:0/2726437456 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd480156a90 con 0x7fd480163f30 2024-04-25T10:59:44.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.640+0000 7fd486020640 1 -- 172.21.15.69:0/2726437456 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd48006e750 con 0x7fd480163f30 2024-04-25T10:59:44.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.640+0000 7fd467fff640 1 -- 172.21.15.69:0/2726437456 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fd47405f910 con 0x7fd480163f30 2024-04-25T10:59:44.641 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-25T10:59:44.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.642+0000 7fd486020640 1 -- 172.21.15.69:0/2726437456 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd46c0744b0 msgr2=0x7fd46c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:44.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.642+0000 7fd486020640 1 --2- 172.21.15.69:0/2726437456 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd46c0744b0 0x7fd46c0769c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd470034000 tx=0x7fd47002e000 comp rx=0 tx=0).stop 2024-04-25T10:59:44.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.642+0000 7fd486020640 1 -- 172.21.15.69:0/2726437456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd480163f30 msgr2=0x7fd48015b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:44.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.642+0000 7fd486020640 1 --2- 172.21.15.69:0/2726437456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd480163f30 0x7fd48015b860 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd47400e870 tx=0x7fd47400ed30 comp rx=0 tx=0).stop 2024-04-25T10:59:44.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.642+0000 7fd486020640 1 -- 172.21.15.69:0/2726437456 shutdown_connections 2024-04-25T10:59:44.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.642+0000 7fd486020640 1 --2- 172.21.15.69:0/2726437456 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd46c0744b0 0x7fd46c0769c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:44.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.642+0000 7fd486020640 1 --2- 172.21.15.69:0/2726437456 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd480164ad0 0x7fd48015bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:44.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.642+0000 7fd486020640 1 --2- 172.21.15.69:0/2726437456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd480163f30 0x7fd48015b860 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:44.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.642+0000 7fd486020640 1 --2- 172.21.15.69:0/2726437456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4800d1f30 0x7fd48015b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:44.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.642+0000 7fd486020640 1 -- 172.21.15.69:0/2726437456 >> 172.21.15.69:0/2726437456 conn(0x7fd4800d3aa0 msgr2=0x7fd4800d2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:44.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.642+0000 7fd486020640 1 -- 172.21.15.69:0/2726437456 shutdown_connections 2024-04-25T10:59:44.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.642+0000 7fd486020640 1 -- 172.21.15.69:0/2726437456 wait complete. 2024-04-25T10:59:44.654 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_cap_flush.TestCapFlush.test_replay_create' 2024-04-25T10:59:44.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.762+0000 7f6d8e57a640 1 Processor -- start 2024-04-25T10:59:44.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.762+0000 7f6d8e57a640 1 -- start start 2024-04-25T10:59:44.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.763+0000 7f6d8e57a640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6d88164b00 0x7f6d880b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:44.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.763+0000 7f6d8e57a640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f6d8815df80 0x7f6d880b80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:44.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.763+0000 7f6d8e57a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d88160890 0x7f6d88160ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:44.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.763+0000 7f6d8e57a640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f6d88164670 con 0x7f6d8815df80 2024-04-25T10:59:44.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.763+0000 7f6d8e57a640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f6d881615f0 con 0x7f6d88164b00 2024-04-25T10:59:44.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.763+0000 7f6d8e57a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6d88164020 con 0x7f6d88160890 2024-04-25T10:59:44.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.763+0000 7f6d877fe640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f6d8815df80 0x7f6d880b80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:44.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.763+0000 7f6d877fe640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f6d8815df80 0x7f6d880b80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43916/0 (socket says 172.21.15.69:43916) 2024-04-25T10:59:44.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.763+0000 7f6d877fe640 1 -- 172.21.15.69:0/3136039905 learned_addr learned my addr 172.21.15.69:0/3136039905 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:44.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.763+0000 7f6d8cb17640 1 --2- 172.21.15.69:0/3136039905 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d88160890 0x7f6d88160ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:44.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.763+0000 7f6d87fff640 1 --2- 172.21.15.69:0/3136039905 >> v2:172.21.15.146:3300/0 conn(0x7f6d88164b00 0x7f6d880b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:44.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.763+0000 7f6d877fe640 1 -- 172.21.15.69:0/3136039905 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d88160890 msgr2=0x7f6d88160ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:44.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.763+0000 7f6d877fe640 1 --2- 172.21.15.69:0/3136039905 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d88160890 0x7f6d88160ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:44.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.763+0000 7f6d877fe640 1 -- 172.21.15.69:0/3136039905 >> v2:172.21.15.146:3300/0 conn(0x7f6d88164b00 msgr2=0x7f6d880b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:44.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.764+0000 7f6d877fe640 1 --2- 172.21.15.69:0/3136039905 >> v2:172.21.15.146:3300/0 conn(0x7f6d88164b00 0x7f6d880b7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:44.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.764+0000 7f6d877fe640 1 -- 172.21.15.69:0/3136039905 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d88164300 con 0x7f6d8815df80 2024-04-25T10:59:44.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.764+0000 7f6d87fff640 1 --2- 172.21.15.69:0/3136039905 >> v2:172.21.15.146:3300/0 conn(0x7f6d88164b00 0x7f6d880b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:44.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.764+0000 7f6d877fe640 1 --2- 172.21.15.69:0/3136039905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d8815df80 0x7f6d880b80c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6d8808e410 tx=0x7f6d7c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1632b3d6dcaf5992 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:44.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.764+0000 7f6d86ffd640 1 -- 172.21.15.69:0/3136039905 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c037070 con 0x7f6d8815df80 2024-04-25T10:59:44.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.764+0000 7f6d86ffd640 1 -- 172.21.15.69:0/3136039905 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d7c032440 con 0x7f6d8815df80 2024-04-25T10:59:44.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.764+0000 7f6d86ffd640 1 -- 172.21.15.69:0/3136039905 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c032810 con 0x7f6d8815df80 2024-04-25T10:59:44.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.765+0000 7f6d8e57a640 1 -- 172.21.15.69:0/3136039905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d8815df80 msgr2=0x7f6d880b80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:44.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.765+0000 7f6d8e57a640 1 --2- 172.21.15.69:0/3136039905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d8815df80 0x7f6d880b80c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6d8808e410 tx=0x7f6d7c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:59:44.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.765+0000 7f6d8e57a640 1 -- 172.21.15.69:0/3136039905 shutdown_connections 2024-04-25T10:59:44.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.765+0000 7f6d8e57a640 1 --2- 172.21.15.69:0/3136039905 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d88160890 0x7f6d88160ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:44.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.765+0000 7f6d8e57a640 1 --2- 172.21.15.69:0/3136039905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d8815df80 0x7f6d880b80c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:44.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.765+0000 7f6d8e57a640 1 --2- 172.21.15.69:0/3136039905 >> v2:172.21.15.146:3300/0 conn(0x7f6d88164b00 0x7f6d880b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:44.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.765+0000 7f6d8e57a640 1 -- 172.21.15.69:0/3136039905 >> 172.21.15.69:0/3136039905 conn(0x7f6d880d2bc0 msgr2=0x7f6d880d2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:44.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.765+0000 7f6d8e57a640 1 -- 172.21.15.69:0/3136039905 shutdown_connections 2024-04-25T10:59:44.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.765+0000 7f6d8e57a640 1 -- 172.21.15.69:0/3136039905 wait complete. 2024-04-25T10:59:44.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.767+0000 7f6d8e57a640 1 Processor -- start 2024-04-25T10:59:44.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.767+0000 7f6d8e57a640 1 -- start start 2024-04-25T10:59:44.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.768+0000 7f6d8e57a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d8815df80 0x7f6d8814b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:44.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.768+0000 7f6d8e57a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d88160890 0x7f6d881ea040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:44.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.768+0000 7f6d8e57a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d88164b00 0x7f6d880c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:44.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.768+0000 7f6d8e57a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6d880b4a00 con 0x7f6d88164b00 2024-04-25T10:59:44.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.768+0000 7f6d8e57a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6d880b4880 con 0x7f6d8815df80 2024-04-25T10:59:44.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.768+0000 7f6d8e57a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6d880b4b80 con 0x7f6d88160890 2024-04-25T10:59:44.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.768+0000 7f6d8cb17640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d88164b00 0x7f6d880c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:44.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.768+0000 7f6d8cb17640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d88164b00 0x7f6d880c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43920/0 (socket says 172.21.15.69:43920) 2024-04-25T10:59:44.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.768+0000 7f6d8cb17640 1 -- 172.21.15.69:0/2792532899 learned_addr learned my addr 172.21.15.69:0/2792532899 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:44.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.769+0000 7f6d877fe640 1 --2- 172.21.15.69:0/2792532899 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d88160890 0x7f6d881ea040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:44.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.769+0000 7f6d87fff640 1 --2- 172.21.15.69:0/2792532899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d8815df80 0x7f6d8814b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:44.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.769+0000 7f6d8cb17640 1 -- 172.21.15.69:0/2792532899 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d88160890 msgr2=0x7f6d881ea040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:44.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.769+0000 7f6d8cb17640 1 --2- 172.21.15.69:0/2792532899 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d88160890 0x7f6d881ea040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:44.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.769+0000 7f6d8cb17640 1 -- 172.21.15.69:0/2792532899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d8815df80 msgr2=0x7f6d8814b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:44.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.769+0000 7f6d8cb17640 1 --2- 172.21.15.69:0/2792532899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d8815df80 0x7f6d8814b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:44.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.769+0000 7f6d8cb17640 1 -- 172.21.15.69:0/2792532899 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d881489c0 con 0x7f6d88164b00 2024-04-25T10:59:44.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.769+0000 7f6d8cb17640 1 --2- 172.21.15.69:0/2792532899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d88164b00 0x7f6d880c8cc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6d8000cbb0 tx=0x7f6d8000a4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:44.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.769+0000 7f6d84ff9640 1 -- 172.21.15.69:0/2792532899 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d80015070 con 0x7f6d88164b00 2024-04-25T10:59:44.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.769+0000 7f6d84ff9640 1 -- 172.21.15.69:0/2792532899 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d8000abd0 con 0x7f6d88164b00 2024-04-25T10:59:44.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.769+0000 7f6d8e57a640 1 -- 172.21.15.69:0/2792532899 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d880c9a40 con 0x7f6d88164b00 2024-04-25T10:59:44.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.769+0000 7f6d84ff9640 1 -- 172.21.15.69:0/2792532899 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8000d040 con 0x7f6d88164b00 2024-04-25T10:59:44.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.769+0000 7f6d8e57a640 1 -- 172.21.15.69:0/2792532899 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d8814a200 con 0x7f6d88164b00 2024-04-25T10:59:44.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.770+0000 7f6d8e57a640 1 -- 172.21.15.69:0/2792532899 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d880c97b0 con 0x7f6d88164b00 2024-04-25T10:59:44.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.771+0000 7f6d84ff9640 1 -- 172.21.15.69:0/2792532899 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6d80011410 con 0x7f6d88164b00 2024-04-25T10:59:44.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.771+0000 7f6d84ff9640 1 --2- 172.21.15.69:0/2792532899 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f6d740744b0 0x7f6d740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:44.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.771+0000 7f6d84ff9640 1 -- 172.21.15.69:0/2792532899 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6d80094cd0 con 0x7f6d88164b00 2024-04-25T10:59:44.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.771+0000 7f6d87fff640 1 --2- 172.21.15.69:0/2792532899 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f6d740744b0 0x7f6d740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:44.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.772+0000 7f6d87fff640 1 --2- 172.21.15.69:0/2792532899 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f6d740744b0 0x7f6d740769c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6d7800a950 tx=0x7f6d78008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:44.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:44.776+0000 7f6d84ff9640 1 -- 172.21.15.69:0/2792532899 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6d8005f4f0 con 0x7f6d88164b00 2024-04-25T10:59:45.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.017+0000 7f6d8e57a640 1 -- 172.21.15.69:0/2792532899 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_cap_flush.TestCapFlush.test_replay_create"]} v 0) -- 0x7f6d88050740 con 0x7f6d88164b00 2024-04-25T10:59:45.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.717+0000 7f6d84ff9640 1 -- 172.21.15.69:0/2792532899 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_cap_flush.TestCapFlush.test_replay_create"]}]=0 v48) ==== 141+0+0 (secure 0 0 0) 0x7f6d8005edf0 con 0x7f6d88164b00 2024-04-25T10:59:45.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.720+0000 7f6d8e57a640 1 -- 172.21.15.69:0/2792532899 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f6d740744b0 msgr2=0x7f6d740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:45.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.720+0000 7f6d8e57a640 1 --2- 172.21.15.69:0/2792532899 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f6d740744b0 0x7f6d740769c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6d7800a950 tx=0x7f6d78008040 comp rx=0 tx=0).stop 2024-04-25T10:59:45.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.720+0000 7f6d8e57a640 1 -- 172.21.15.69:0/2792532899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d88164b00 msgr2=0x7f6d880c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:45.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.720+0000 7f6d8e57a640 1 --2- 172.21.15.69:0/2792532899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d88164b00 0x7f6d880c8cc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6d8000cbb0 tx=0x7f6d8000a4c0 comp rx=0 tx=0).stop 2024-04-25T10:59:45.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.720+0000 7f6d8e57a640 1 -- 172.21.15.69:0/2792532899 shutdown_connections 2024-04-25T10:59:45.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.720+0000 7f6d8e57a640 1 --2- 172.21.15.69:0/2792532899 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f6d740744b0 0x7f6d740769c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:45.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.720+0000 7f6d8e57a640 1 --2- 172.21.15.69:0/2792532899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d88164b00 0x7f6d880c8cc0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:45.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.720+0000 7f6d8e57a640 1 --2- 172.21.15.69:0/2792532899 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d88160890 0x7f6d881ea040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:45.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.720+0000 7f6d8e57a640 1 --2- 172.21.15.69:0/2792532899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d8815df80 0x7f6d8814b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:45.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.720+0000 7f6d8e57a640 1 -- 172.21.15.69:0/2792532899 >> 172.21.15.69:0/2792532899 conn(0x7f6d880d2bc0 msgr2=0x7f6d8815edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:45.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.720+0000 7f6d8e57a640 1 -- 172.21.15.69:0/2792532899 shutdown_connections 2024-04-25T10:59:45.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.720+0000 7f6d8e57a640 1 -- 172.21.15.69:0/2792532899 wait complete. 2024-04-25T10:59:45.735 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:59:45.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.851+0000 7f9788c34640 1 Processor -- start 2024-04-25T10:59:45.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.851+0000 7f9788c34640 1 -- start start 2024-04-25T10:59:45.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.853+0000 7f9788c34640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97840b6900 0x7f97840b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:45.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.853+0000 7f9788c34640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f97840d20c0 0x7f9784161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:45.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.853+0000 7f9788c34640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97840ce670 con 0x7f97840b4e70 2024-04-25T10:59:45.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.853+0000 7f9788c34640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f978408d770 con 0x7f97840d20c0 2024-04-25T10:59:45.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.853+0000 7f9788c34640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f97840ce980 con 0x7f97840b6900 2024-04-25T10:59:45.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.853+0000 7f9782d9d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f97840d20c0 0x7f9784161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:45.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.853+0000 7f978259c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97840b6900 0x7f97840b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:45.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.853+0000 7f9781d9b640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f97840b4e70 0x7f97840d19b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.69:48700/0 (socket says 172.21.15.69:48700) 2024-04-25T10:59:45.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.853+0000 7f9781d9b640 1 -- 172.21.15.69:0/2951611589 learned_addr learned my addr 172.21.15.69:0/2951611589 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:45.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.853+0000 7f9782d9d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f97840d20c0 0x7f9784161df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.69:41572/0 (socket says 172.21.15.69:41572) 2024-04-25T10:59:45.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.853+0000 7f9782d9d640 1 -- 172.21.15.69:0/2951611589 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97840b6900 msgr2=0x7f97840b8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:45.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.853+0000 7f9782d9d640 1 --2- 172.21.15.69:0/2951611589 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97840b6900 0x7f97840b8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:45.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.853+0000 7f9782d9d640 1 -- 172.21.15.69:0/2951611589 >> v1:172.21.15.146:6789/0 conn(0x7f97840b4e70 legacy=0x7f97840d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:45.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.853+0000 7f9782d9d640 1 -- 172.21.15.69:0/2951611589 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f978408d9a0 con 0x7f97840d20c0 2024-04-25T10:59:45.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.853+0000 7f978259c640 1 --2- 172.21.15.69:0/2951611589 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97840b6900 0x7f97840b8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:59:45.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.854+0000 7f9782d9d640 1 --2- 172.21.15.69:0/2951611589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97840d20c0 0x7f9784161df0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f977c009d90 tx=0x7f977c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9722c864fc9c02c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:45.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.854+0000 7f978159a640 1 -- 172.21.15.69:0/2951611589 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c007740 con 0x7f97840d20c0 2024-04-25T10:59:45.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.854+0000 7f978159a640 1 -- 172.21.15.69:0/2951611589 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f977c0078e0 con 0x7f97840d20c0 2024-04-25T10:59:45.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.854+0000 7f978159a640 1 -- 172.21.15.69:0/2951611589 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c007cb0 con 0x7f97840d20c0 2024-04-25T10:59:45.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.855+0000 7f9788c34640 1 -- 172.21.15.69:0/2951611589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97840d20c0 msgr2=0x7f9784161df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:45.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.855+0000 7f9788c34640 1 --2- 172.21.15.69:0/2951611589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97840d20c0 0x7f9784161df0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f977c009d90 tx=0x7f977c02a660 comp rx=0 tx=0).stop 2024-04-25T10:59:45.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.855+0000 7f9788c34640 1 -- 172.21.15.69:0/2951611589 shutdown_connections 2024-04-25T10:59:45.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.855+0000 7f9788c34640 1 --2- 172.21.15.69:0/2951611589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97840d20c0 0x7f9784161df0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:45.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.855+0000 7f9788c34640 1 --2- 172.21.15.69:0/2951611589 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97840b6900 0x7f97840b8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:45.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.855+0000 7f9788c34640 1 -- 172.21.15.69:0/2951611589 >> 172.21.15.69:0/2951611589 conn(0x7f97840d0530 msgr2=0x7f97840d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:45.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.855+0000 7f9788c34640 1 -- 172.21.15.69:0/2951611589 shutdown_connections 2024-04-25T10:59:45.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.855+0000 7f9788c34640 1 -- 172.21.15.69:0/2951611589 wait complete. 2024-04-25T10:59:45.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.857+0000 7f9788c34640 1 Processor -- start 2024-04-25T10:59:45.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.857+0000 7f9788c34640 1 -- start start 2024-04-25T10:59:45.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.858+0000 7f9788c34640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97840b4e70 0x7f978415b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:45.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.858+0000 7f9788c34640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97840b6900 0x7f978415f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:45.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.858+0000 7f9788c34640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97840d20c0 0x7f978415fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:45.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.858+0000 7f9788c34640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f97840b5280 con 0x7f97840b4e70 2024-04-25T10:59:45.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.858+0000 7f9788c34640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f978408df40 con 0x7f97840b6900 2024-04-25T10:59:45.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.858+0000 7f9788c34640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f97840b5400 con 0x7f97840d20c0 2024-04-25T10:59:45.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.859+0000 7f978259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97840b4e70 0x7f978415b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:45.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.859+0000 7f978259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97840b4e70 0x7f978415b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54956/0 (socket says 172.21.15.69:54956) 2024-04-25T10:59:45.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.859+0000 7f978259c640 1 -- 172.21.15.69:0/1419480779 learned_addr learned my addr 172.21.15.69:0/1419480779 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:45.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.859+0000 7f9781d9b640 1 --2- 172.21.15.69:0/1419480779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97840b6900 0x7f978415f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:45.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.859+0000 7f9782d9d640 1 --2- 172.21.15.69:0/1419480779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97840d20c0 0x7f978415fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:45.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.859+0000 7f978259c640 1 -- 172.21.15.69:0/1419480779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97840d20c0 msgr2=0x7f978415fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:45.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.859+0000 7f978259c640 1 --2- 172.21.15.69:0/1419480779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97840d20c0 0x7f978415fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:45.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.859+0000 7f978259c640 1 -- 172.21.15.69:0/1419480779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97840b6900 msgr2=0x7f978415f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:45.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.859+0000 7f978259c640 1 --2- 172.21.15.69:0/1419480779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97840b6900 0x7f978415f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:45.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.859+0000 7f978259c640 1 -- 172.21.15.69:0/1419480779 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9784160e30 con 0x7f97840b4e70 2024-04-25T10:59:45.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.859+0000 7f9782d9d640 1 --2- 172.21.15.69:0/1419480779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97840d20c0 0x7f978415fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:45.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.859+0000 7f978259c640 1 --2- 172.21.15.69:0/1419480779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97840b4e70 0x7f978415b000 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f977400e990 tx=0x7f977400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:45.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.859+0000 7f9772ffd640 1 -- 172.21.15.69:0/1419480779 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9774017070 con 0x7f97840b4e70 2024-04-25T10:59:45.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.859+0000 7f9772ffd640 1 -- 172.21.15.69:0/1419480779 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f977400c9e0 con 0x7f97840b4e70 2024-04-25T10:59:45.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.859+0000 7f9788c34640 1 -- 172.21.15.69:0/1419480779 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f978415ea50 con 0x7f97840b4e70 2024-04-25T10:59:45.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.859+0000 7f9772ffd640 1 -- 172.21.15.69:0/1419480779 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977400cdb0 con 0x7f97840b4e70 2024-04-25T10:59:45.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.859+0000 7f9788c34640 1 -- 172.21.15.69:0/1419480779 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9784158310 con 0x7f97840b4e70 2024-04-25T10:59:45.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.860+0000 7f9788c34640 1 -- 172.21.15.69:0/1419480779 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9784050740 con 0x7f97840b4e70 2024-04-25T10:59:45.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.861+0000 7f9772ffd640 1 -- 172.21.15.69:0/1419480779 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f977401a070 con 0x7f97840b4e70 2024-04-25T10:59:45.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.861+0000 7f9772ffd640 1 --2- 172.21.15.69:0/1419480779 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f97640744e0 0x7f97640769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:45.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.861+0000 7f9772ffd640 1 -- 172.21.15.69:0/1419480779 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9774095460 con 0x7f97840b4e70 2024-04-25T10:59:45.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.861+0000 7f9781d9b640 1 --2- 172.21.15.69:0/1419480779 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f97640744e0 0x7f97640769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:45.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.862+0000 7f9781d9b640 1 --2- 172.21.15.69:0/1419480779 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f97640744e0 0x7f97640769f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f97841f8080 tx=0x7f9778007e80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:45.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:45.866+0000 7f9772ffd640 1 -- 172.21.15.69:0/1419480779 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f977405f520 con 0x7f97840b4e70 2024-04-25T10:59:46.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.041+0000 7f9788c34640 1 -- 172.21.15.69:0/1419480779 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f9784071000 con 0x7f97840b4e70 2024-04-25T10:59:46.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.045+0000 7f9772ffd640 1 -- 172.21.15.69:0/1419480779 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f9774063820 con 0x7f97840b4e70 2024-04-25T10:59:46.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.047+0000 7f9788c34640 1 -- 172.21.15.69:0/1419480779 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f97640744e0 msgr2=0x7f97640769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:46.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.047+0000 7f9788c34640 1 --2- 172.21.15.69:0/1419480779 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f97640744e0 0x7f97640769f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f97841f8080 tx=0x7f9778007e80 comp rx=0 tx=0).stop 2024-04-25T10:59:46.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.047+0000 7f9788c34640 1 -- 172.21.15.69:0/1419480779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97840b4e70 msgr2=0x7f978415b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:46.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.047+0000 7f9788c34640 1 --2- 172.21.15.69:0/1419480779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97840b4e70 0x7f978415b000 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f977400e990 tx=0x7f977400ee50 comp rx=0 tx=0).stop 2024-04-25T10:59:46.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.047+0000 7f9788c34640 1 -- 172.21.15.69:0/1419480779 shutdown_connections 2024-04-25T10:59:46.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.047+0000 7f9788c34640 1 --2- 172.21.15.69:0/1419480779 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f97640744e0 0x7f97640769f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:46.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.047+0000 7f9788c34640 1 --2- 172.21.15.69:0/1419480779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97840d20c0 0x7f978415fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:46.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.047+0000 7f9788c34640 1 --2- 172.21.15.69:0/1419480779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97840b6900 0x7f978415f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:46.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.047+0000 7f9788c34640 1 --2- 172.21.15.69:0/1419480779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97840b4e70 0x7f978415b000 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:46.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.047+0000 7f9788c34640 1 -- 172.21.15.69:0/1419480779 >> 172.21.15.69:0/1419480779 conn(0x7f97840d0530 msgr2=0x7f97841544e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:46.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.047+0000 7f9788c34640 1 -- 172.21.15.69:0/1419480779 shutdown_connections 2024-04-25T10:59:46.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.047+0000 7f9788c34640 1 -- 172.21.15.69:0/1419480779 wait complete. 2024-04-25T10:59:46.059 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:59:46.059 DEBUG:teuthology.orchestra.run.smithi069:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:59:46.096 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:59:46.140 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:59:46.140 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:59:46.140 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:59:46.156 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:59:46.156 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:59:46.211 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:59:46.212 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:59:46.266 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:59:46.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.412+0000 7fc83a74d640 1 Processor -- start 2024-04-25T10:59:46.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.412+0000 7fc83a74d640 1 -- start start 2024-04-25T10:59:46.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.413+0000 7fc83a74d640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fc83415cbe0 0x7fc8340d1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:46.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.413+0000 7fc83a74d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc8340b0690 0x7fc8340d2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:46.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.413+0000 7fc83a74d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc83406de70 con 0x7fc8340b76d0 2024-04-25T10:59:46.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.413+0000 7fc83a74d640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fc83406e150 con 0x7fc83415cbe0 2024-04-25T10:59:46.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.413+0000 7fc83a74d640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fc8340af1f0 con 0x7fc8340b0690 2024-04-25T10:59:46.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.413+0000 7fc833fff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fc83415cbe0 0x7fc8340d1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:46.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.413+0000 7fc833fff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fc83415cbe0 0x7fc8340d1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54972/0 (socket says 172.21.15.69:54972) 2024-04-25T10:59:46.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.413+0000 7fc833fff640 1 -- 172.21.15.69:0/2417126739 learned_addr learned my addr 172.21.15.69:0/2417126739 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:46.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.414+0000 7fc833fff640 1 -- 172.21.15.69:0/2417126739 >> v2:172.21.15.146:3300/0 conn(0x7fc8340b0690 msgr2=0x7fc8340d2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:46.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.414+0000 7fc838cea640 1 --2- 172.21.15.69:0/2417126739 >> v2:172.21.15.146:3300/0 conn(0x7fc8340b0690 0x7fc8340d2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:46.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.414+0000 7fc833fff640 1 --2- 172.21.15.69:0/2417126739 >> v2:172.21.15.146:3300/0 conn(0x7fc8340b0690 0x7fc8340d2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:46.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.414+0000 7fc833fff640 1 -- 172.21.15.69:0/2417126739 >> v1:172.21.15.146:6789/0 conn(0x7fc8340b76d0 legacy=0x7fc83406c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:46.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.414+0000 7fc833fff640 1 -- 172.21.15.69:0/2417126739 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8340b1800 con 0x7fc83415cbe0 2024-04-25T10:59:46.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.414+0000 7fc833fff640 1 --2- 172.21.15.69:0/2417126739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc83415cbe0 0x7fc8340d1ff0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc834073910 tx=0x7fc82402c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40ffa7d19d01b34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:46.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.415+0000 7fc832ffd640 1 -- 172.21.15.69:0/2417126739 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82403a070 con 0x7fc83415cbe0 2024-04-25T10:59:46.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.415+0000 7fc832ffd640 1 -- 172.21.15.69:0/2417126739 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc824032480 con 0x7fc83415cbe0 2024-04-25T10:59:46.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.415+0000 7fc832ffd640 1 -- 172.21.15.69:0/2417126739 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc824032850 con 0x7fc83415cbe0 2024-04-25T10:59:46.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.415+0000 7fc83a74d640 1 -- 172.21.15.69:0/2417126739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc83415cbe0 msgr2=0x7fc8340d1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:46.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.415+0000 7fc83a74d640 1 --2- 172.21.15.69:0/2417126739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc83415cbe0 0x7fc8340d1ff0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc834073910 tx=0x7fc82402c6b0 comp rx=0 tx=0).stop 2024-04-25T10:59:46.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.415+0000 7fc83a74d640 1 -- 172.21.15.69:0/2417126739 shutdown_connections 2024-04-25T10:59:46.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.415+0000 7fc83a74d640 1 --2- 172.21.15.69:0/2417126739 >> v2:172.21.15.146:3300/0 conn(0x7fc8340b0690 0x7fc8340d2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:46.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.415+0000 7fc83a74d640 1 --2- 172.21.15.69:0/2417126739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc83415cbe0 0x7fc8340d1ff0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:46.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.415+0000 7fc83a74d640 1 -- 172.21.15.69:0/2417126739 >> 172.21.15.69:0/2417126739 conn(0x7fc83409b7d0 msgr2=0x7fc8340b6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:46.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.415+0000 7fc83a74d640 1 -- 172.21.15.69:0/2417126739 shutdown_connections 2024-04-25T10:59:46.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.415+0000 7fc83a74d640 1 -- 172.21.15.69:0/2417126739 wait complete. 2024-04-25T10:59:46.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.417+0000 7fc83a74d640 1 Processor -- start 2024-04-25T10:59:46.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.417+0000 7fc83a74d640 1 -- start start 2024-04-25T10:59:46.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.418+0000 7fc83a74d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8340b0690 0x7fc8340b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:46.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.418+0000 7fc83a74d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8340b76d0 0x7fc8340b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:46.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.418+0000 7fc83a74d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc8341e5ab0 0x7fc8340b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:46.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.418+0000 7fc83a74d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc83406da00 con 0x7fc8340b76d0 2024-04-25T10:59:46.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.418+0000 7fc83a74d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc83406d880 con 0x7fc8340b0690 2024-04-25T10:59:46.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.419+0000 7fc83a74d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc83406db80 con 0x7fc8341e5ab0 2024-04-25T10:59:46.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.419+0000 7fc8337fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8340b76d0 0x7fc8340b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:46.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.419+0000 7fc8337fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8340b76d0 0x7fc8340b3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54986/0 (socket says 172.21.15.69:54986) 2024-04-25T10:59:46.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.419+0000 7fc8337fe640 1 -- 172.21.15.69:0/303888315 learned_addr learned my addr 172.21.15.69:0/303888315 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:46.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.419+0000 7fc833fff640 1 --2- 172.21.15.69:0/303888315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8340b0690 0x7fc8340b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:46.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.419+0000 7fc8337fe640 1 -- 172.21.15.69:0/303888315 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc8341e5ab0 msgr2=0x7fc8340b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:46.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.419+0000 7fc8337fe640 1 --2- 172.21.15.69:0/303888315 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc8341e5ab0 0x7fc8340b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:46.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.419+0000 7fc8337fe640 1 -- 172.21.15.69:0/303888315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8340b0690 msgr2=0x7fc8340b34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:46.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.419+0000 7fc8337fe640 1 --2- 172.21.15.69:0/303888315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8340b0690 0x7fc8340b34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:46.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.419+0000 7fc8337fe640 1 -- 172.21.15.69:0/303888315 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc83415fc30 con 0x7fc8340b76d0 2024-04-25T10:59:46.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.419+0000 7fc8337fe640 1 --2- 172.21.15.69:0/303888315 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8340b76d0 0x7fc8340b3a20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc82800b420 tx=0x7fc82800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:46.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.419+0000 7fc830ff9640 1 -- 172.21.15.69:0/303888315 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc828011830 con 0x7fc8340b76d0 2024-04-25T10:59:46.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.419+0000 7fc830ff9640 1 -- 172.21.15.69:0/303888315 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc828016070 con 0x7fc8340b76d0 2024-04-25T10:59:46.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.419+0000 7fc830ff9640 1 -- 172.21.15.69:0/303888315 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc828011c10 con 0x7fc8340b76d0 2024-04-25T10:59:46.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.419+0000 7fc83a74d640 1 -- 172.21.15.69:0/303888315 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc83415f950 con 0x7fc8340b76d0 2024-04-25T10:59:46.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.419+0000 7fc83a74d640 1 -- 172.21.15.69:0/303888315 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8340b2ce0 con 0x7fc8340b76d0 2024-04-25T10:59:46.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.420+0000 7fc83a74d640 1 -- 172.21.15.69:0/303888315 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc83414e160 con 0x7fc8340b76d0 2024-04-25T10:59:46.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.421+0000 7fc830ff9640 1 -- 172.21.15.69:0/303888315 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc82801b460 con 0x7fc8340b76d0 2024-04-25T10:59:46.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.421+0000 7fc830ff9640 1 --2- 172.21.15.69:0/303888315 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc8200744e0 0x7fc8200769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:46.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.421+0000 7fc830ff9640 1 -- 172.21.15.69:0/303888315 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc828094e00 con 0x7fc8340b76d0 2024-04-25T10:59:46.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.421+0000 7fc833fff640 1 --2- 172.21.15.69:0/303888315 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc8200744e0 0x7fc8200769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:46.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.422+0000 7fc833fff640 1 --2- 172.21.15.69:0/303888315 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc8200744e0 0x7fc8200769f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc824037000 tx=0x7fc824030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:46.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.426+0000 7fc830ff9640 1 -- 172.21.15.69:0/303888315 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc82805f6b0 con 0x7fc8340b76d0 2024-04-25T10:59:46.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.601+0000 7fc83a74d640 1 -- 172.21.15.69:0/303888315 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc834050740 con 0x7fc8340b76d0 2024-04-25T10:59:46.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.603+0000 7fc830ff9640 1 -- 172.21.15.69:0/303888315 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7fc82805efb0 con 0x7fc8340b76d0 2024-04-25T10:59:46.604 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:46.604 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":20,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T10:59:20.630151+0000","last_up_change":"2024-04-25T10:59:05.546839+0000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:59:06.422674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:59:16.727980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:59:17.938486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6825","nonce":1876101065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6827","nonce":1876101065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6831","nonce":1876101065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6829","nonce":1876101065}]},"public_addr":"172.21.15.69:6825/1876101065","cluster_addr":"172.21.15.69:6827/1876101065","heartbeat_back_addr":"172.21.15.69:6831/1876101065","heartbeat_front_addr":"172.21.15.69:6829/1876101065","state":["exists","up"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6801","nonce":1806438563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6803","nonce":1806438563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6807","nonce":1806438563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6805","nonce":1806438563}]},"public_addr":"172.21.15.69:6801/1806438563","cluster_addr":"172.21.15.69:6803/1806438563","heartbeat_back_addr":"172.21.15.69:6807/1806438563","heartbeat_front_addr":"172.21.15.69:6805/1806438563","state":["exists","up"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6817","nonce":1593617983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6819","nonce":1593617983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6823","nonce":1593617983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6821","nonce":1593617983}]},"public_addr":"172.21.15.69:6817/1593617983","cluster_addr":"172.21.15.69:6819/1593617983","heartbeat_back_addr":"172.21.15.69:6823/1593617983","heartbeat_front_addr":"172.21.15.69:6821/1593617983","state":["exists","up"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6809","nonce":3950661294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6811","nonce":3950661294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6815","nonce":3950661294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6813","nonce":3950661294}]},"public_addr":"172.21.15.69:6809/3950661294","cluster_addr":"172.21.15.69:6811/3950661294","heartbeat_back_addr":"172.21.15.69:6815/3950661294","heartbeat_front_addr":"172.21.15.69:6813/3950661294","state":["exists","up"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6801","nonce":2508929872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6803","nonce":2508929872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6807","nonce":2508929872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6805","nonce":2508929872}]},"public_addr":"172.21.15.146:6801/2508929872","cluster_addr":"172.21.15.146:6803/2508929872","heartbeat_back_addr":"172.21.15.146:6807/2508929872","heartbeat_front_addr":"172.21.15.146:6805/2508929872","state":["exists","up"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6811","nonce":1310911138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6816","nonce":1310911138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6823","nonce":1310911138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6821","nonce":1310911138}]},"public_addr":"172.21.15.146:6811/1310911138","cluster_addr":"172.21.15.146:6816/1310911138","heartbeat_back_addr":"172.21.15.146:6823/1310911138","heartbeat_front_addr":"172.21.15.146:6821/1310911138","state":["exists","up"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6810","nonce":634688021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6813","nonce":634688021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6820","nonce":634688021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6817","nonce":634688021}]},"public_addr":"172.21.15.146:6810/634688021","cluster_addr":"172.21.15.146:6813/634688021","heartbeat_back_addr":"172.21.15.146:6820/634688021","heartbeat_front_addr":"172.21.15.146:6817/634688021","state":["exists","up"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6825","nonce":3814215935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6827","nonce":3814215935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6831","nonce":3814215935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6829","nonce":3814215935}]},"public_addr":"172.21.15.146:6825/3814215935","cluster_addr":"172.21.15.146:6827/3814215935","heartbeat_back_addr":"172.21.15.146:6831/3814215935","heartbeat_front_addr":"172.21.15.146:6829/3814215935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.820200+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.850189+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.764560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.866248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.807897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.869881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.782310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.827131+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:59:46.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.605+0000 7fc83a74d640 1 -- 172.21.15.69:0/303888315 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc8200744e0 msgr2=0x7fc8200769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:46.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.605+0000 7fc83a74d640 1 --2- 172.21.15.69:0/303888315 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc8200744e0 0x7fc8200769f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc824037000 tx=0x7fc824030000 comp rx=0 tx=0).stop 2024-04-25T10:59:46.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.605+0000 7fc83a74d640 1 -- 172.21.15.69:0/303888315 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8340b76d0 msgr2=0x7fc8340b3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:46.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.605+0000 7fc83a74d640 1 --2- 172.21.15.69:0/303888315 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8340b76d0 0x7fc8340b3a20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc82800b420 tx=0x7fc82800b8e0 comp rx=0 tx=0).stop 2024-04-25T10:59:46.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.605+0000 7fc83a74d640 1 -- 172.21.15.69:0/303888315 shutdown_connections 2024-04-25T10:59:46.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.605+0000 7fc83a74d640 1 --2- 172.21.15.69:0/303888315 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc8200744e0 0x7fc8200769f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:46.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.605+0000 7fc83a74d640 1 --2- 172.21.15.69:0/303888315 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc8341e5ab0 0x7fc8340b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:46.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.605+0000 7fc83a74d640 1 --2- 172.21.15.69:0/303888315 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8340b76d0 0x7fc8340b3a20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:46.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.605+0000 7fc83a74d640 1 --2- 172.21.15.69:0/303888315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8340b0690 0x7fc8340b34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:46.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.605+0000 7fc83a74d640 1 -- 172.21.15.69:0/303888315 >> 172.21.15.69:0/303888315 conn(0x7fc83409b7d0 msgr2=0x7fc8340d0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:46.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.605+0000 7fc83a74d640 1 -- 172.21.15.69:0/303888315 shutdown_connections 2024-04-25T10:59:46.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.605+0000 7fc83a74d640 1 -- 172.21.15.69:0/303888315 wait complete. 2024-04-25T10:59:46.617 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:59:06.422674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:59:16.727980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:59:17.938486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:59:46.617 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:59:46.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.727+0000 7fe31157d640 1 Processor -- start 2024-04-25T10:59:46.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.727+0000 7fe31157d640 1 -- start start 2024-04-25T10:59:46.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.728+0000 7fe31157d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe30c06c8f0 0x7fe30c0b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:46.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.728+0000 7fe31157d640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe30c0b0c20 0x7fe30c0b1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:46.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.728+0000 7fe31157d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe30c0b32f0 con 0x7fe30c075e80 2024-04-25T10:59:46.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.728+0000 7fe31157d640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fe30c0b0250 con 0x7fe30c0b0c20 2024-04-25T10:59:46.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.728+0000 7fe31157d640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fe30c0b35d0 con 0x7fe30c06c8f0 2024-04-25T10:59:46.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.728+0000 7fe30b7fe640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe30c0b0c20 0x7fe30c0b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:46.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.728+0000 7fe30b7fe640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe30c0b0c20 0x7fe30c0b1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54998/0 (socket says 172.21.15.69:54998) 2024-04-25T10:59:46.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.728+0000 7fe30b7fe640 1 -- 172.21.15.69:0/2444194855 learned_addr learned my addr 172.21.15.69:0/2444194855 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:46.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.728+0000 7fe30a7fc640 1 --2- 172.21.15.69:0/2444194855 >> v2:172.21.15.146:3300/0 conn(0x7fe30c06c8f0 0x7fe30c0b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:46.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.728+0000 7fe30b7fe640 1 -- 172.21.15.69:0/2444194855 >> v2:172.21.15.146:3300/0 conn(0x7fe30c06c8f0 msgr2=0x7fe30c0b06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:46.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.728+0000 7fe30b7fe640 1 --2- 172.21.15.69:0/2444194855 >> v2:172.21.15.146:3300/0 conn(0x7fe30c06c8f0 0x7fe30c0b06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:46.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.728+0000 7fe30b7fe640 1 -- 172.21.15.69:0/2444194855 >> v1:172.21.15.146:6789/0 conn(0x7fe30c075e80 legacy=0x7fe30c0cff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:46.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.728+0000 7fe30b7fe640 1 -- 172.21.15.69:0/2444194855 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe30c0b38b0 con 0x7fe30c0b0c20 2024-04-25T10:59:46.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.729+0000 7fe30b7fe640 1 --2- 172.21.15.69:0/2444194855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe30c0b0c20 0x7fe30c0b1340 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe30c0cfee0 tx=0x7fe30402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8ccb964ae19114 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:46.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.729+0000 7fe309ffb640 1 -- 172.21.15.69:0/2444194855 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe304037070 con 0x7fe30c0b0c20 2024-04-25T10:59:46.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.729+0000 7fe309ffb640 1 -- 172.21.15.69:0/2444194855 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe304032440 con 0x7fe30c0b0c20 2024-04-25T10:59:46.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.729+0000 7fe309ffb640 1 -- 172.21.15.69:0/2444194855 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe304032810 con 0x7fe30c0b0c20 2024-04-25T10:59:46.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.730+0000 7fe31157d640 1 -- 172.21.15.69:0/2444194855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe30c0b0c20 msgr2=0x7fe30c0b1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:46.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.730+0000 7fe31157d640 1 --2- 172.21.15.69:0/2444194855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe30c0b0c20 0x7fe30c0b1340 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe30c0cfee0 tx=0x7fe30402a270 comp rx=0 tx=0).stop 2024-04-25T10:59:46.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.730+0000 7fe31157d640 1 -- 172.21.15.69:0/2444194855 shutdown_connections 2024-04-25T10:59:46.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.730+0000 7fe31157d640 1 --2- 172.21.15.69:0/2444194855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe30c0b0c20 0x7fe30c0b1340 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:46.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.730+0000 7fe31157d640 1 --2- 172.21.15.69:0/2444194855 >> v2:172.21.15.146:3300/0 conn(0x7fe30c06c8f0 0x7fe30c0b06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:46.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.730+0000 7fe31157d640 1 -- 172.21.15.69:0/2444194855 >> 172.21.15.69:0/2444194855 conn(0x7fe30c09b7d0 msgr2=0x7fe30c0b52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:46.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.730+0000 7fe31157d640 1 -- 172.21.15.69:0/2444194855 shutdown_connections 2024-04-25T10:59:46.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.730+0000 7fe31157d640 1 -- 172.21.15.69:0/2444194855 wait complete. 2024-04-25T10:59:46.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.732+0000 7fe31157d640 1 Processor -- start 2024-04-25T10:59:46.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.732+0000 7fe31157d640 1 -- start start 2024-04-25T10:59:46.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.733+0000 7fe31157d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe30c06c8f0 0x7fe30c15a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:46.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.733+0000 7fe31157d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe30c075e80 0x7fe30c15d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:46.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.733+0000 7fe31157d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe30c0b0c20 0x7fe30c0ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:46.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.733+0000 7fe31157d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe30c0d0640 con 0x7fe30c075e80 2024-04-25T10:59:46.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.733+0000 7fe31157d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe30c0b3b90 con 0x7fe30c0b0c20 2024-04-25T10:59:46.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.733+0000 7fe31157d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe30c0cd700 con 0x7fe30c06c8f0 2024-04-25T10:59:46.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.733+0000 7fe30a7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe30c075e80 0x7fe30c15d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:46.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.733+0000 7fe30a7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe30c075e80 0x7fe30c15d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55008/0 (socket says 172.21.15.69:55008) 2024-04-25T10:59:46.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.733+0000 7fe30a7fc640 1 -- 172.21.15.69:0/3003865990 learned_addr learned my addr 172.21.15.69:0/3003865990 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:46.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.733+0000 7fe30b7fe640 1 --2- 172.21.15.69:0/3003865990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe30c0b0c20 0x7fe30c0ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:46.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.733+0000 7fe30affd640 1 --2- 172.21.15.69:0/3003865990 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe30c06c8f0 0x7fe30c15a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:46.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.734+0000 7fe30a7fc640 1 -- 172.21.15.69:0/3003865990 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe30c06c8f0 msgr2=0x7fe30c15a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:46.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.734+0000 7fe30a7fc640 1 --2- 172.21.15.69:0/3003865990 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe30c06c8f0 0x7fe30c15a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:46.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.734+0000 7fe30a7fc640 1 -- 172.21.15.69:0/3003865990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe30c0b0c20 msgr2=0x7fe30c0ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:46.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.734+0000 7fe30a7fc640 1 --2- 172.21.15.69:0/3003865990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe30c0b0c20 0x7fe30c0ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:46.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.734+0000 7fe30a7fc640 1 -- 172.21.15.69:0/3003865990 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe30c15c580 con 0x7fe30c075e80 2024-04-25T10:59:46.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.734+0000 7fe30b7fe640 1 --2- 172.21.15.69:0/3003865990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe30c0b0c20 0x7fe30c0ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:46.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.734+0000 7fe30affd640 1 --2- 172.21.15.69:0/3003865990 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe30c06c8f0 0x7fe30c15a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:46.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.734+0000 7fe30a7fc640 1 --2- 172.21.15.69:0/3003865990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe30c075e80 0x7fe30c15d620 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe30000d5f0 tx=0x7fe30000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:46.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.734+0000 7fe2f37fe640 1 -- 172.21.15.69:0/3003865990 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3000125f0 con 0x7fe30c075e80 2024-04-25T10:59:46.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.734+0000 7fe2f37fe640 1 -- 172.21.15.69:0/3003865990 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe300019070 con 0x7fe30c075e80 2024-04-25T10:59:46.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.734+0000 7fe31157d640 1 -- 172.21.15.69:0/3003865990 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe30c15a710 con 0x7fe30c075e80 2024-04-25T10:59:46.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.734+0000 7fe2f37fe640 1 -- 172.21.15.69:0/3003865990 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3000129d0 con 0x7fe30c075e80 2024-04-25T10:59:46.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.734+0000 7fe31157d640 1 -- 172.21.15.69:0/3003865990 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe30c15cb40 con 0x7fe30c075e80 2024-04-25T10:59:46.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.735+0000 7fe31157d640 1 -- 172.21.15.69:0/3003865990 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe30c050740 con 0x7fe30c075e80 2024-04-25T10:59:46.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.736+0000 7fe2f37fe640 1 -- 172.21.15.69:0/3003865990 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe300012b70 con 0x7fe30c075e80 2024-04-25T10:59:46.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.736+0000 7fe2f37fe640 1 --2- 172.21.15.69:0/3003865990 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe2f80744b0 0x7fe2f80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:46.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.736+0000 7fe2f37fe640 1 -- 172.21.15.69:0/3003865990 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe3000956b0 con 0x7fe30c075e80 2024-04-25T10:59:46.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.736+0000 7fe30affd640 1 --2- 172.21.15.69:0/3003865990 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe2f80744b0 0x7fe2f80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:46.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.737+0000 7fe30affd640 1 --2- 172.21.15.69:0/3003865990 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe2f80744b0 0x7fe2f80769c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc001d90 tx=0x7fe2fc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:46.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.741+0000 7fe2f37fe640 1 -- 172.21.15.69:0/3003865990 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe30005fe40 con 0x7fe30c075e80 2024-04-25T10:59:46.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.918+0000 7fe31157d640 1 -- 172.21.15.69:0/3003865990 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe30c068f60 con 0x7fe30c075e80 2024-04-25T10:59:46.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.918+0000 7fe2f37fe640 1 -- 172.21.15.69:0/3003865990 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fe30005f740 con 0x7fe30c075e80 2024-04-25T10:59:46.919 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-25T10:59:46.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.920+0000 7fe31157d640 1 -- 172.21.15.69:0/3003865990 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe2f80744b0 msgr2=0x7fe2f80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:46.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.920+0000 7fe31157d640 1 --2- 172.21.15.69:0/3003865990 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe2f80744b0 0x7fe2f80769c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc001d90 tx=0x7fe2fc009040 comp rx=0 tx=0).stop 2024-04-25T10:59:46.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.920+0000 7fe31157d640 1 -- 172.21.15.69:0/3003865990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe30c075e80 msgr2=0x7fe30c15d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:46.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.920+0000 7fe31157d640 1 --2- 172.21.15.69:0/3003865990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe30c075e80 0x7fe30c15d620 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe30000d5f0 tx=0x7fe30000dab0 comp rx=0 tx=0).stop 2024-04-25T10:59:46.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.920+0000 7fe31157d640 1 -- 172.21.15.69:0/3003865990 shutdown_connections 2024-04-25T10:59:46.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.920+0000 7fe31157d640 1 --2- 172.21.15.69:0/3003865990 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe2f80744b0 0x7fe2f80769c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:46.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.920+0000 7fe31157d640 1 --2- 172.21.15.69:0/3003865990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe30c0b0c20 0x7fe30c0ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:46.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.920+0000 7fe31157d640 1 --2- 172.21.15.69:0/3003865990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe30c075e80 0x7fe30c15d620 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:46.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.920+0000 7fe31157d640 1 --2- 172.21.15.69:0/3003865990 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe30c06c8f0 0x7fe30c15a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:46.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.920+0000 7fe31157d640 1 -- 172.21.15.69:0/3003865990 >> 172.21.15.69:0/3003865990 conn(0x7fe30c09b7d0 msgr2=0x7fe30c0d0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:46.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.920+0000 7fe31157d640 1 -- 172.21.15.69:0/3003865990 shutdown_connections 2024-04-25T10:59:46.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:46.920+0000 7fe31157d640 1 -- 172.21.15.69:0/3003865990 wait complete. 2024-04-25T10:59:46.932 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:59:47.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.042+0000 7faebf7ee640 1 Processor -- start 2024-04-25T10:59:47.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.042+0000 7faebf7ee640 1 -- start start 2024-04-25T10:59:47.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.044+0000 7faebf7ee640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7faeb80b0670 0x7faeb80b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:47.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.044+0000 7faebf7ee640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faeb80cf520 0x7faeb80cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:47.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.044+0000 7faebf7ee640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7faeb80d16e0 0x7faeb80d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:47.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.044+0000 7faebf7ee640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7faeb80d2030 con 0x7faeb80b0670 2024-04-25T10:59:47.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.044+0000 7faebf7ee640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7faeb80ce7e0 con 0x7faeb80d16e0 2024-04-25T10:59:47.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.044+0000 7faebf7ee640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7faeb80cea60 con 0x7faeb80cf520 2024-04-25T10:59:47.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.044+0000 7faebd58a640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7faeb80b0670 0x7faeb80b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:47.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.044+0000 7faebd58a640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7faeb80b0670 0x7faeb80b0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55024/0 (socket says 172.21.15.69:55024) 2024-04-25T10:59:47.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.044+0000 7faebd58a640 1 -- 172.21.15.69:0/3027740931 learned_addr learned my addr 172.21.15.69:0/3027740931 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:47.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.044+0000 7faebdd8b640 1 --2- 172.21.15.69:0/3027740931 >> v2:172.21.15.146:3300/0 conn(0x7faeb80d16e0 0x7faeb80d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:47.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.044+0000 7faebcd89640 1 --2- 172.21.15.69:0/3027740931 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faeb80cf520 0x7faeb80cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:47.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.044+0000 7faebd58a640 1 -- 172.21.15.69:0/3027740931 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faeb80cf520 msgr2=0x7faeb80cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:47.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.044+0000 7faebd58a640 1 --2- 172.21.15.69:0/3027740931 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faeb80cf520 0x7faeb80cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:47.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.044+0000 7faebd58a640 1 -- 172.21.15.69:0/3027740931 >> v2:172.21.15.146:3300/0 conn(0x7faeb80d16e0 msgr2=0x7faeb80d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:47.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.044+0000 7faebd58a640 1 --2- 172.21.15.69:0/3027740931 >> v2:172.21.15.146:3300/0 conn(0x7faeb80d16e0 0x7faeb80d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:47.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.044+0000 7faebd58a640 1 -- 172.21.15.69:0/3027740931 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb815f370 con 0x7faeb80b0670 2024-04-25T10:59:47.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.045+0000 7faebd58a640 1 --2- 172.21.15.69:0/3027740931 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faeb80b0670 0x7faeb80b0a80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7faeb8161da0 tx=0x7faeb402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187974e539ee9c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:47.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.046+0000 7faeaffff640 1 -- 172.21.15.69:0/3027740931 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb4037070 con 0x7faeb80b0670 2024-04-25T10:59:47.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.046+0000 7faeaffff640 1 -- 172.21.15.69:0/3027740931 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faeb4032440 con 0x7faeb80b0670 2024-04-25T10:59:47.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.046+0000 7faeaffff640 1 -- 172.21.15.69:0/3027740931 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb4032810 con 0x7faeb80b0670 2024-04-25T10:59:47.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.047+0000 7faebf7ee640 1 -- 172.21.15.69:0/3027740931 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faeb80b0670 msgr2=0x7faeb80b0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:47.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.047+0000 7faebf7ee640 1 --2- 172.21.15.69:0/3027740931 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faeb80b0670 0x7faeb80b0a80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7faeb8161da0 tx=0x7faeb402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:59:47.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.047+0000 7faebf7ee640 1 -- 172.21.15.69:0/3027740931 shutdown_connections 2024-04-25T10:59:47.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.047+0000 7faebf7ee640 1 --2- 172.21.15.69:0/3027740931 >> v2:172.21.15.146:3300/0 conn(0x7faeb80d16e0 0x7faeb80d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:47.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.047+0000 7faebf7ee640 1 --2- 172.21.15.69:0/3027740931 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faeb80cf520 0x7faeb80cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:47.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.047+0000 7faebf7ee640 1 --2- 172.21.15.69:0/3027740931 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faeb80b0670 0x7faeb80b0a80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:47.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.047+0000 7faebf7ee640 1 -- 172.21.15.69:0/3027740931 >> 172.21.15.69:0/3027740931 conn(0x7faeb80d3a80 msgr2=0x7faeb80d3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:47.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.047+0000 7faebf7ee640 1 -- 172.21.15.69:0/3027740931 shutdown_connections 2024-04-25T10:59:47.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.047+0000 7faebf7ee640 1 -- 172.21.15.69:0/3027740931 wait complete. 2024-04-25T10:59:47.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.049+0000 7faebf7ee640 1 Processor -- start 2024-04-25T10:59:47.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.049+0000 7faebf7ee640 1 -- start start 2024-04-25T10:59:47.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.050+0000 7faebf7ee640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faeb80b0670 0x7faeb8152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:47.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.050+0000 7faebf7ee640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faeb80cf520 0x7faeb80caea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:47.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.050+0000 7faebf7ee640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faeb80d16e0 0x7faeb8152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:47.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.050+0000 7faebf7ee640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7faeb80cf000 con 0x7faeb80cf520 2024-04-25T10:59:47.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.050+0000 7faebf7ee640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7faeb80ced20 con 0x7faeb80b0670 2024-04-25T10:59:47.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.050+0000 7faebf7ee640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7faeb80d21f0 con 0x7faeb80d16e0 2024-04-25T10:59:47.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.050+0000 7faebcd89640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faeb80cf520 0x7faeb80caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:47.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.050+0000 7faebcd89640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faeb80cf520 0x7faeb80caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55030/0 (socket says 172.21.15.69:55030) 2024-04-25T10:59:47.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.050+0000 7faebcd89640 1 -- 172.21.15.69:0/2914704475 learned_addr learned my addr 172.21.15.69:0/2914704475 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:47.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.051+0000 7faebd58a640 1 --2- 172.21.15.69:0/2914704475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faeb80b0670 0x7faeb8152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:47.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.051+0000 7faebdd8b640 1 --2- 172.21.15.69:0/2914704475 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faeb80d16e0 0x7faeb8152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:47.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.051+0000 7faebcd89640 1 -- 172.21.15.69:0/2914704475 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faeb80d16e0 msgr2=0x7faeb8152e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:47.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.051+0000 7faebcd89640 1 --2- 172.21.15.69:0/2914704475 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faeb80d16e0 0x7faeb8152e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:47.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.051+0000 7faebcd89640 1 -- 172.21.15.69:0/2914704475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faeb80b0670 msgr2=0x7faeb8152800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:47.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.051+0000 7faebcd89640 1 --2- 172.21.15.69:0/2914704475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faeb80b0670 0x7faeb8152800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:47.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.051+0000 7faebcd89640 1 -- 172.21.15.69:0/2914704475 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb8160c30 con 0x7faeb80cf520 2024-04-25T10:59:47.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.051+0000 7faebd58a640 1 --2- 172.21.15.69:0/2914704475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faeb80b0670 0x7faeb8152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:47.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.051+0000 7faebcd89640 1 --2- 172.21.15.69:0/2914704475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faeb80cf520 0x7faeb80caea0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faea800e870 tx=0x7faea800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:47.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.051+0000 7faeadffb640 1 -- 172.21.15.69:0/2914704475 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea8017070 con 0x7faeb80cf520 2024-04-25T10:59:47.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.051+0000 7faeadffb640 1 -- 172.21.15.69:0/2914704475 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faea800ee60 con 0x7faeb80cf520 2024-04-25T10:59:47.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.051+0000 7faebf7ee640 1 -- 172.21.15.69:0/2914704475 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeb8160f10 con 0x7faeb80cf520 2024-04-25T10:59:47.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.051+0000 7faeadffb640 1 -- 172.21.15.69:0/2914704475 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea800cb80 con 0x7faeb80cf520 2024-04-25T10:59:47.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.051+0000 7faebf7ee640 1 -- 172.21.15.69:0/2914704475 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeb80b2e10 con 0x7faeb80cf520 2024-04-25T10:59:47.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.052+0000 7faebf7ee640 1 -- 172.21.15.69:0/2914704475 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeb8050740 con 0x7faeb80cf520 2024-04-25T10:59:47.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.053+0000 7faeadffb640 1 -- 172.21.15.69:0/2914704475 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7faea801a070 con 0x7faeb80cf520 2024-04-25T10:59:47.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.053+0000 7faeadffb640 1 --2- 172.21.15.69:0/2914704475 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fae9c0744b0 0x7fae9c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:47.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.053+0000 7faeadffb640 1 -- 172.21.15.69:0/2914704475 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7faea8095460 con 0x7faeb80cf520 2024-04-25T10:59:47.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.054+0000 7faebd58a640 1 --2- 172.21.15.69:0/2914704475 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fae9c0744b0 0x7fae9c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:47.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.054+0000 7faebd58a640 1 --2- 172.21.15.69:0/2914704475 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fae9c0744b0 0x7fae9c0769c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7faeb4034000 tx=0x7faeb402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:47.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.058+0000 7faeadffb640 1 -- 172.21.15.69:0/2914704475 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faea805fbf0 con 0x7faeb80cf520 2024-04-25T10:59:47.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.231+0000 7faebf7ee640 1 -- 172.21.15.69:0/2914704475 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7faeb81609c0 con 0x7faeb80cf520 2024-04-25T10:59:47.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.231+0000 7faeadffb640 1 -- 172.21.15.69:0/2914704475 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7faea805f4f0 con 0x7faeb80cf520 2024-04-25T10:59:47.232 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-25T10:59:47.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.233+0000 7faebf7ee640 1 -- 172.21.15.69:0/2914704475 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fae9c0744b0 msgr2=0x7fae9c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:47.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.233+0000 7faebf7ee640 1 --2- 172.21.15.69:0/2914704475 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fae9c0744b0 0x7fae9c0769c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7faeb4034000 tx=0x7faeb402e000 comp rx=0 tx=0).stop 2024-04-25T10:59:47.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.233+0000 7faebf7ee640 1 -- 172.21.15.69:0/2914704475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faeb80cf520 msgr2=0x7faeb80caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:47.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.233+0000 7faebf7ee640 1 --2- 172.21.15.69:0/2914704475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faeb80cf520 0x7faeb80caea0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faea800e870 tx=0x7faea800ed30 comp rx=0 tx=0).stop 2024-04-25T10:59:47.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.233+0000 7faebf7ee640 1 -- 172.21.15.69:0/2914704475 shutdown_connections 2024-04-25T10:59:47.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.233+0000 7faebf7ee640 1 --2- 172.21.15.69:0/2914704475 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fae9c0744b0 0x7fae9c0769c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:47.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.233+0000 7faebf7ee640 1 --2- 172.21.15.69:0/2914704475 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faeb80d16e0 0x7faeb8152e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:47.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.233+0000 7faebf7ee640 1 --2- 172.21.15.69:0/2914704475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faeb80cf520 0x7faeb80caea0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:47.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.233+0000 7faebf7ee640 1 --2- 172.21.15.69:0/2914704475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faeb80b0670 0x7faeb8152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:47.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.233+0000 7faebf7ee640 1 -- 172.21.15.69:0/2914704475 >> 172.21.15.69:0/2914704475 conn(0x7faeb80d3a80 msgr2=0x7faeb80b5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:47.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.233+0000 7faebf7ee640 1 -- 172.21.15.69:0/2914704475 shutdown_connections 2024-04-25T10:59:47.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.233+0000 7faebf7ee640 1 -- 172.21.15.69:0/2914704475 wait complete. 2024-04-25T10:59:47.245 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:59:47.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.354+0000 7ff139651640 1 Processor -- start 2024-04-25T10:59:47.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.354+0000 7ff139651640 1 -- start start 2024-04-25T10:59:47.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.355+0000 7ff139651640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff1340ccff0 0x7ff1340d28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:47.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.355+0000 7ff139651640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1340d2e30 0x7ff1340b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:47.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.355+0000 7ff139651640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff13406e6c0 con 0x7ff13406e0e0 2024-04-25T10:59:47.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.355+0000 7ff139651640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7ff1340d3240 con 0x7ff1340ccff0 2024-04-25T10:59:47.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.355+0000 7ff139651640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff1340b3e90 con 0x7ff1340d2e30 2024-04-25T10:59:47.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.356+0000 7ff1337fe640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1340d2e30 0x7ff1340b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:47.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.356+0000 7ff1327fc640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff1340ccff0 0x7ff1340d28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:47.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.356+0000 7ff1337fe640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1340d2e30 0x7ff1340b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.69:54558/0 (socket says 172.21.15.69:54558) 2024-04-25T10:59:47.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.356+0000 7ff1327fc640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff1340ccff0 0x7ff1340d28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.69:41630/0 (socket says 172.21.15.69:41630) 2024-04-25T10:59:47.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.356+0000 7ff132ffd640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7ff13406e0e0 0x7ff13406eaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.69:48726/0 (socket says 172.21.15.69:48726) 2024-04-25T10:59:47.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.356+0000 7ff1337fe640 1 -- 172.21.15.69:0/1725299139 learned_addr learned my addr 172.21.15.69:0/1725299139 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:47.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.356+0000 7ff1327fc640 1 -- 172.21.15.69:0/1725299139 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1340d2e30 msgr2=0x7ff1340b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:47.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.356+0000 7ff1327fc640 1 --2- 172.21.15.69:0/1725299139 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1340d2e30 0x7ff1340b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:47.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.356+0000 7ff1327fc640 1 -- 172.21.15.69:0/1725299139 >> v1:172.21.15.146:6789/0 conn(0x7ff13406e0e0 legacy=0x7ff13406eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:47.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.356+0000 7ff1327fc640 1 -- 172.21.15.69:0/1725299139 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1340cd400 con 0x7ff1340ccff0 2024-04-25T10:59:47.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.356+0000 7ff1337fe640 1 --2- 172.21.15.69:0/1725299139 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1340d2e30 0x7ff1340b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:59:47.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.357+0000 7ff1327fc640 1 --2- 172.21.15.69:0/1725299139 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1340ccff0 0x7ff1340d28f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff11c0097a0 tx=0x7ff11c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14071837eb71fe62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:47.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.357+0000 7ff131ffb640 1 -- 172.21.15.69:0/1725299139 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c037070 con 0x7ff1340ccff0 2024-04-25T10:59:47.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.357+0000 7ff131ffb640 1 -- 172.21.15.69:0/1725299139 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff11c032440 con 0x7ff1340ccff0 2024-04-25T10:59:47.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.357+0000 7ff131ffb640 1 -- 172.21.15.69:0/1725299139 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c032810 con 0x7ff1340ccff0 2024-04-25T10:59:47.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.358+0000 7ff139651640 1 -- 172.21.15.69:0/1725299139 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1340ccff0 msgr2=0x7ff1340d28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:47.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.358+0000 7ff139651640 1 --2- 172.21.15.69:0/1725299139 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1340ccff0 0x7ff1340d28f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff11c0097a0 tx=0x7ff11c02a180 comp rx=0 tx=0).stop 2024-04-25T10:59:47.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.358+0000 7ff139651640 1 -- 172.21.15.69:0/1725299139 shutdown_connections 2024-04-25T10:59:47.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.358+0000 7ff139651640 1 --2- 172.21.15.69:0/1725299139 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1340d2e30 0x7ff1340b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:47.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.358+0000 7ff139651640 1 --2- 172.21.15.69:0/1725299139 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1340ccff0 0x7ff1340d28f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:47.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.358+0000 7ff139651640 1 -- 172.21.15.69:0/1725299139 >> 172.21.15.69:0/1725299139 conn(0x7ff13409b7d0 msgr2=0x7ff13409bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:47.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.358+0000 7ff139651640 1 -- 172.21.15.69:0/1725299139 shutdown_connections 2024-04-25T10:59:47.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.358+0000 7ff139651640 1 -- 172.21.15.69:0/1725299139 wait complete. 2024-04-25T10:59:47.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.360+0000 7ff139651640 1 Processor -- start 2024-04-25T10:59:47.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.360+0000 7ff139651640 1 -- start start 2024-04-25T10:59:47.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.361+0000 7ff139651640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff13406e0e0 0x7ff1340cc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:47.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.361+0000 7ff139651640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1340ccff0 0x7ff134157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:47.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.361+0000 7ff139651640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1340d2e30 0x7ff1340ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:47.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.361+0000 7ff139651640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff1340b29d0 con 0x7ff13406e0e0 2024-04-25T10:59:47.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.361+0000 7ff139651640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff1340b26f0 con 0x7ff1340d2e30 2024-04-25T10:59:47.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.361+0000 7ff139651640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff1340b4130 con 0x7ff1340ccff0 2024-04-25T10:59:47.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.361+0000 7ff132ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff13406e0e0 0x7ff1340cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:47.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.361+0000 7ff132ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff13406e0e0 0x7ff1340cc190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55032/0 (socket says 172.21.15.69:55032) 2024-04-25T10:59:47.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.361+0000 7ff132ffd640 1 -- 172.21.15.69:0/1063541010 learned_addr learned my addr 172.21.15.69:0/1063541010 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:47.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.361+0000 7ff1327fc640 1 --2- 172.21.15.69:0/1063541010 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1340ccff0 0x7ff134157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:47.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.361+0000 7ff1337fe640 1 --2- 172.21.15.69:0/1063541010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1340d2e30 0x7ff1340ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:47.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.361+0000 7ff132ffd640 1 -- 172.21.15.69:0/1063541010 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1340ccff0 msgr2=0x7ff134157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:47.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.361+0000 7ff132ffd640 1 --2- 172.21.15.69:0/1063541010 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1340ccff0 0x7ff134157b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:47.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.361+0000 7ff132ffd640 1 -- 172.21.15.69:0/1063541010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1340d2e30 msgr2=0x7ff1340ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:47.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.361+0000 7ff132ffd640 1 --2- 172.21.15.69:0/1063541010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1340d2e30 0x7ff1340ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:47.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.361+0000 7ff132ffd640 1 -- 172.21.15.69:0/1063541010 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1340c9bf0 con 0x7ff13406e0e0 2024-04-25T10:59:47.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.361+0000 7ff1327fc640 1 --2- 172.21.15.69:0/1063541010 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1340ccff0 0x7ff134157b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:47.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.362+0000 7ff132ffd640 1 --2- 172.21.15.69:0/1063541010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff13406e0e0 0x7ff1340cc190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00caa0 tx=0x7ff12c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:47.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.362+0000 7ff12b7fe640 1 -- 172.21.15.69:0/1063541010 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c017070 con 0x7ff13406e0e0 2024-04-25T10:59:47.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.362+0000 7ff12b7fe640 1 -- 172.21.15.69:0/1063541010 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff12c00a7f0 con 0x7ff13406e0e0 2024-04-25T10:59:47.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.362+0000 7ff12b7fe640 1 -- 172.21.15.69:0/1063541010 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c00ac40 con 0x7ff13406e0e0 2024-04-25T10:59:47.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.362+0000 7ff139651640 1 -- 172.21.15.69:0/1063541010 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1340cc9e0 con 0x7ff13406e0e0 2024-04-25T10:59:47.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.362+0000 7ff139651640 1 -- 172.21.15.69:0/1063541010 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1340c9e90 con 0x7ff13406e0e0 2024-04-25T10:59:47.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.363+0000 7ff139651640 1 -- 172.21.15.69:0/1063541010 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff134050740 con 0x7ff13406e0e0 2024-04-25T10:59:47.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.363+0000 7ff12b7fe640 1 -- 172.21.15.69:0/1063541010 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff12c01a070 con 0x7ff13406e0e0 2024-04-25T10:59:47.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.364+0000 7ff12b7fe640 1 --2- 172.21.15.69:0/1063541010 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff1140744e0 0x7ff1140769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:47.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.364+0000 7ff12b7fe640 1 -- 172.21.15.69:0/1063541010 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff12c094be0 con 0x7ff13406e0e0 2024-04-25T10:59:47.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.364+0000 7ff1327fc640 1 --2- 172.21.15.69:0/1063541010 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff1140744e0 0x7ff1140769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:47.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.364+0000 7ff1327fc640 1 --2- 172.21.15.69:0/1063541010 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff1140744e0 0x7ff1140769f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff11c034000 tx=0x7ff11c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:47.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.368+0000 7ff12b7fe640 1 -- 172.21.15.69:0/1063541010 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff12c05f370 con 0x7ff13406e0e0 2024-04-25T10:59:47.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.542+0000 7ff139651640 1 -- 172.21.15.69:0/1063541010 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff1340d2830 con 0x7ff13406e0e0 2024-04-25T10:59:47.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.542+0000 7ff12b7fe640 1 -- 172.21.15.69:0/1063541010 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7ff12c0673f0 con 0x7ff13406e0e0 2024-04-25T10:59:47.543 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-25T10:59:47.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.544+0000 7ff139651640 1 -- 172.21.15.69:0/1063541010 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff1140744e0 msgr2=0x7ff1140769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:47.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.544+0000 7ff139651640 1 --2- 172.21.15.69:0/1063541010 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff1140744e0 0x7ff1140769f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff11c034000 tx=0x7ff11c02e000 comp rx=0 tx=0).stop 2024-04-25T10:59:47.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.544+0000 7ff139651640 1 -- 172.21.15.69:0/1063541010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff13406e0e0 msgr2=0x7ff1340cc190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:47.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.544+0000 7ff139651640 1 --2- 172.21.15.69:0/1063541010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff13406e0e0 0x7ff1340cc190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00caa0 tx=0x7ff12c00cf60 comp rx=0 tx=0).stop 2024-04-25T10:59:47.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.544+0000 7ff139651640 1 -- 172.21.15.69:0/1063541010 shutdown_connections 2024-04-25T10:59:47.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.544+0000 7ff139651640 1 --2- 172.21.15.69:0/1063541010 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff1140744e0 0x7ff1140769f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:47.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.544+0000 7ff139651640 1 --2- 172.21.15.69:0/1063541010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1340d2e30 0x7ff1340ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:47.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.544+0000 7ff139651640 1 --2- 172.21.15.69:0/1063541010 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1340ccff0 0x7ff134157b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:47.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.544+0000 7ff139651640 1 --2- 172.21.15.69:0/1063541010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff13406e0e0 0x7ff1340cc190 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:47.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.544+0000 7ff139651640 1 -- 172.21.15.69:0/1063541010 >> 172.21.15.69:0/1063541010 conn(0x7ff13409b7d0 msgr2=0x7ff1340b76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:47.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.544+0000 7ff139651640 1 -- 172.21.15.69:0/1063541010 shutdown_connections 2024-04-25T10:59:47.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.544+0000 7ff139651640 1 -- 172.21.15.69:0/1063541010 wait complete. 2024-04-25T10:59:47.556 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:59:47.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.663+0000 7fdfa04e7640 1 Processor -- start 2024-04-25T10:59:47.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.663+0000 7fdfa04e7640 1 -- start start 2024-04-25T10:59:47.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.665+0000 7fdfa04e7640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fdf980b7ef0 0x7fdf980d2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:47.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.665+0000 7fdfa04e7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf980cd6d0 0x7fdf980cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:47.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.665+0000 7fdfa04e7640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf980b4350 con 0x7fdf9806ea80 2024-04-25T10:59:47.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.665+0000 7fdfa04e7640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fdf980d3830 con 0x7fdf980b7ef0 2024-04-25T10:59:47.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.665+0000 7fdfa04e7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fdf980d2f90 con 0x7fdf980cd6d0 2024-04-25T10:59:47.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.665+0000 7fdf9da82640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fdf980b7ef0 0x7fdf980d2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:47.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.665+0000 7fdf9da82640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fdf980b7ef0 0x7fdf980d2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55034/0 (socket says 172.21.15.69:55034) 2024-04-25T10:59:47.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.665+0000 7fdf9da82640 1 -- 172.21.15.69:0/4098803891 learned_addr learned my addr 172.21.15.69:0/4098803891 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:47.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.665+0000 7fdf9ea84640 1 --2- 172.21.15.69:0/4098803891 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf980cd6d0 0x7fdf980cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:47.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.665+0000 7fdf9da82640 1 -- 172.21.15.69:0/4098803891 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf980cd6d0 msgr2=0x7fdf980cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:47.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.665+0000 7fdf9da82640 1 --2- 172.21.15.69:0/4098803891 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf980cd6d0 0x7fdf980cdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:47.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.665+0000 7fdf9da82640 1 -- 172.21.15.69:0/4098803891 >> v1:172.21.15.146:6789/0 conn(0x7fdf9806ea80 legacy=0x7fdf980b77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:47.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.665+0000 7fdf9da82640 1 -- 172.21.15.69:0/4098803891 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf980d3270 con 0x7fdf980b7ef0 2024-04-25T10:59:47.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.666+0000 7fdf9da82640 1 --2- 172.21.15.69:0/4098803891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf980b7ef0 0x7fdf980d2860 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00bb10 tx=0x7fdf8c02c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f668c01ed4ad69f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:47.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.666+0000 7fdf9d281640 1 -- 172.21.15.69:0/4098803891 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c032790 con 0x7fdf980b7ef0 2024-04-25T10:59:47.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.666+0000 7fdf9d281640 1 -- 172.21.15.69:0/4098803891 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf8c038070 con 0x7fdf980b7ef0 2024-04-25T10:59:47.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.666+0000 7fdf9d281640 1 -- 172.21.15.69:0/4098803891 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c032b50 con 0x7fdf980b7ef0 2024-04-25T10:59:47.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.667+0000 7fdfa04e7640 1 -- 172.21.15.69:0/4098803891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf980b7ef0 msgr2=0x7fdf980d2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:47.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.667+0000 7fdfa04e7640 1 --2- 172.21.15.69:0/4098803891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf980b7ef0 0x7fdf980d2860 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00bb10 tx=0x7fdf8c02c890 comp rx=0 tx=0).stop 2024-04-25T10:59:47.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.667+0000 7fdfa04e7640 1 -- 172.21.15.69:0/4098803891 shutdown_connections 2024-04-25T10:59:47.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.667+0000 7fdfa04e7640 1 --2- 172.21.15.69:0/4098803891 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf980cd6d0 0x7fdf980cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:47.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.667+0000 7fdfa04e7640 1 --2- 172.21.15.69:0/4098803891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf980b7ef0 0x7fdf980d2860 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:47.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.667+0000 7fdfa04e7640 1 -- 172.21.15.69:0/4098803891 >> 172.21.15.69:0/4098803891 conn(0x7fdf9809b7d0 msgr2=0x7fdf9809bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:47.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.667+0000 7fdfa04e7640 1 -- 172.21.15.69:0/4098803891 shutdown_connections 2024-04-25T10:59:47.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.667+0000 7fdfa04e7640 1 -- 172.21.15.69:0/4098803891 wait complete. 2024-04-25T10:59:47.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.669+0000 7fdfa04e7640 1 Processor -- start 2024-04-25T10:59:47.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.669+0000 7fdfa04e7640 1 -- start start 2024-04-25T10:59:47.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.670+0000 7fdfa04e7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf9806ea80 0x7fdf980af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:47.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.670+0000 7fdfa04e7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf980cd6d0 0x7fdf98158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:47.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.670+0000 7fdfa04e7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf98159280 0x7fdf980c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:47.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.670+0000 7fdfa04e7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9815d530 con 0x7fdf98159280 2024-04-25T10:59:47.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.670+0000 7fdfa04e7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdf980cd1b0 con 0x7fdf9806ea80 2024-04-25T10:59:47.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.670+0000 7fdfa04e7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fdf980ce020 con 0x7fdf980cd6d0 2024-04-25T10:59:47.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.670+0000 7fdf9ea84640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf98159280 0x7fdf980c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:47.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.670+0000 7fdf9ea84640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf98159280 0x7fdf980c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55048/0 (socket says 172.21.15.69:55048) 2024-04-25T10:59:47.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.670+0000 7fdf9ea84640 1 -- 172.21.15.69:0/1397774709 learned_addr learned my addr 172.21.15.69:0/1397774709 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:47.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.670+0000 7fdf9e283640 1 --2- 172.21.15.69:0/1397774709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf9806ea80 0x7fdf980af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:47.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.670+0000 7fdf9da82640 1 --2- 172.21.15.69:0/1397774709 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf980cd6d0 0x7fdf98158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:47.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.670+0000 7fdf9ea84640 1 -- 172.21.15.69:0/1397774709 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf980cd6d0 msgr2=0x7fdf98158d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:47.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.670+0000 7fdf9ea84640 1 --2- 172.21.15.69:0/1397774709 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf980cd6d0 0x7fdf98158d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:47.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.670+0000 7fdf9ea84640 1 -- 172.21.15.69:0/1397774709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf9806ea80 msgr2=0x7fdf980af7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:47.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.670+0000 7fdf9ea84640 1 --2- 172.21.15.69:0/1397774709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf9806ea80 0x7fdf980af7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:47.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.670+0000 7fdf9ea84640 1 -- 172.21.15.69:0/1397774709 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf980c70f0 con 0x7fdf98159280 2024-04-25T10:59:47.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.670+0000 7fdf9e283640 1 --2- 172.21.15.69:0/1397774709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf9806ea80 0x7fdf980af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:47.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.670+0000 7fdf9da82640 1 --2- 172.21.15.69:0/1397774709 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf980cd6d0 0x7fdf98158d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:47.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.671+0000 7fdf9ea84640 1 --2- 172.21.15.69:0/1397774709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf98159280 0x7fdf980c8190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdf8800d510 tx=0x7fdf8800d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:47.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.671+0000 7fdf96ffd640 1 -- 172.21.15.69:0/1397774709 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf880125f0 con 0x7fdf98159280 2024-04-25T10:59:47.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.671+0000 7fdf96ffd640 1 -- 172.21.15.69:0/1397774709 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf88019070 con 0x7fdf98159280 2024-04-25T10:59:47.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.671+0000 7fdf96ffd640 1 -- 172.21.15.69:0/1397774709 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf880129f0 con 0x7fdf98159280 2024-04-25T10:59:47.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.671+0000 7fdfa04e7640 1 -- 172.21.15.69:0/1397774709 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf980c73d0 con 0x7fdf98159280 2024-04-25T10:59:47.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.671+0000 7fdfa04e7640 1 -- 172.21.15.69:0/1397774709 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf980c7990 con 0x7fdf98159280 2024-04-25T10:59:47.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.672+0000 7fdfa04e7640 1 -- 172.21.15.69:0/1397774709 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf98050740 con 0x7fdf98159280 2024-04-25T10:59:47.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.672+0000 7fdf96ffd640 1 -- 172.21.15.69:0/1397774709 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdf88012b90 con 0x7fdf98159280 2024-04-25T10:59:47.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.673+0000 7fdf96ffd640 1 --2- 172.21.15.69:0/1397774709 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fdf840744b0 0x7fdf840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:47.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.673+0000 7fdf9e283640 1 --2- 172.21.15.69:0/1397774709 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fdf840744b0 0x7fdf840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:47.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.673+0000 7fdf96ffd640 1 -- 172.21.15.69:0/1397774709 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdf880955c0 con 0x7fdf98159280 2024-04-25T10:59:47.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.673+0000 7fdf9e283640 1 --2- 172.21.15.69:0/1397774709 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fdf840744b0 0x7fdf840769c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdf90001d60 tx=0x7fdf90009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:47.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.678+0000 7fdf96ffd640 1 -- 172.21.15.69:0/1397774709 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdf8805fd50 con 0x7fdf98159280 2024-04-25T10:59:47.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.866+0000 7fdfa04e7640 1 -- 172.21.15.69:0/1397774709 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdf9806f210 con 0x7fdf98159280 2024-04-25T10:59:47.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.866+0000 7fdf96ffd640 1 -- 172.21.15.69:0/1397774709 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7fdf8805f650 con 0x7fdf98159280 2024-04-25T10:59:47.867 INFO:teuthology.orchestra.run.smithi069.stderr:listed 0 entries 2024-04-25T10:59:47.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.868+0000 7fdfa04e7640 1 -- 172.21.15.69:0/1397774709 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fdf840744b0 msgr2=0x7fdf840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:47.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.868+0000 7fdfa04e7640 1 --2- 172.21.15.69:0/1397774709 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fdf840744b0 0x7fdf840769c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdf90001d60 tx=0x7fdf90009040 comp rx=0 tx=0).stop 2024-04-25T10:59:47.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.868+0000 7fdfa04e7640 1 -- 172.21.15.69:0/1397774709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf98159280 msgr2=0x7fdf980c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:47.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.868+0000 7fdfa04e7640 1 --2- 172.21.15.69:0/1397774709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf98159280 0x7fdf980c8190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdf8800d510 tx=0x7fdf8800d9d0 comp rx=0 tx=0).stop 2024-04-25T10:59:47.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.868+0000 7fdfa04e7640 1 -- 172.21.15.69:0/1397774709 shutdown_connections 2024-04-25T10:59:47.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.868+0000 7fdfa04e7640 1 --2- 172.21.15.69:0/1397774709 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fdf840744b0 0x7fdf840769c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:47.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.868+0000 7fdfa04e7640 1 --2- 172.21.15.69:0/1397774709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf98159280 0x7fdf980c8190 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:47.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.868+0000 7fdfa04e7640 1 --2- 172.21.15.69:0/1397774709 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf980cd6d0 0x7fdf98158d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:47.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.868+0000 7fdfa04e7640 1 --2- 172.21.15.69:0/1397774709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf9806ea80 0x7fdf980af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:47.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.868+0000 7fdfa04e7640 1 -- 172.21.15.69:0/1397774709 >> 172.21.15.69:0/1397774709 conn(0x7fdf9809b7d0 msgr2=0x7fdf98070240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:47.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.868+0000 7fdfa04e7640 1 -- 172.21.15.69:0/1397774709 shutdown_connections 2024-04-25T10:59:47.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:47.868+0000 7fdfa04e7640 1 -- 172.21.15.69:0/1397774709 wait complete. 2024-04-25T10:59:47.880 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:59:47.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:47.880 DEBUG:teuthology.orchestra.run.smithi069:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-25T10:59:47.906 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi069.front.sepia.ceph.com 2024-04-25T10:59:47.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:47.907 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:59:47.974 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T10:59:47.974 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:59:47.994 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T10:59:47.994 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:59:48.051 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T10:59:48.052 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:59:48.109 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:59:48.158 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:59:48.161 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:59:48.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.218+0000 7f4a2e630640 1 Processor -- start 2024-04-25T10:59:48.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.218+0000 7f4a2e630640 1 -- start start 2024-04-25T10:59:48.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.219+0000 7f4a2e630640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a2806e0e0 0x7f4a280cff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:48.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.219+0000 7f4a2e630640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4a280b4570 0x7f4a280b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:48.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.219+0000 7f4a2e630640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a2806e4f0 con 0x7f4a2815d130 2024-04-25T10:59:48.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.219+0000 7f4a2e630640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f4a2806f060 con 0x7f4a280b4570 2024-04-25T10:59:48.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.219+0000 7f4a2e630640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4a281611b0 con 0x7f4a2806e0e0 2024-04-25T10:59:48.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.219+0000 7f4a27fff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a2806e0e0 0x7f4a280cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:48.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.219+0000 7f4a2cbcd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4a280b4570 0x7f4a280b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:48.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.219+0000 7f4a2cbcd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4a280b4570 0x7f4a280b4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.69:41662/0 (socket says 172.21.15.69:41662) 2024-04-25T10:59:48.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.219+0000 7f4a27fff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a2806e0e0 0x7f4a280cff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.69:54584/0 (socket says 172.21.15.69:54584) 2024-04-25T10:59:48.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.219+0000 7f4a277fe640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f4a2815d130 0x7f4a28161640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.69:48738/0 (socket says 172.21.15.69:48738) 2024-04-25T10:59:48.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.219+0000 7f4a2cbcd640 1 -- 172.21.15.69:0/3035496801 learned_addr learned my addr 172.21.15.69:0/3035496801 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:48.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.220+0000 7f4a2cbcd640 1 -- 172.21.15.69:0/3035496801 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a2806e0e0 msgr2=0x7f4a280cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:48.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.220+0000 7f4a2cbcd640 1 --2- 172.21.15.69:0/3035496801 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a2806e0e0 0x7f4a280cff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:48.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.220+0000 7f4a2cbcd640 1 -- 172.21.15.69:0/3035496801 >> v1:172.21.15.146:6789/0 conn(0x7f4a2815d130 legacy=0x7f4a28161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:48.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.220+0000 7f4a26ffd640 1 -- 172.21.15.69:0/3035496801 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 120190569 0 0) 0x7f4a1c001110 con 0x7f4a2815d130 2024-04-25T10:59:48.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.220+0000 7f4a2cbcd640 1 -- 172.21.15.69:0/3035496801 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a280b4ec0 con 0x7f4a280b4570 2024-04-25T10:59:48.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.220+0000 7f4a27fff640 1 --2- 172.21.15.69:0/3035496801 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a2806e0e0 0x7f4a280cff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:59:48.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.221+0000 7f4a2cbcd640 1 --2- 172.21.15.69:0/3035496801 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a280b4570 0x7f4a280b4980 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4a2000d150 tx=0x7f4a2002c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f295f8a2ddbec57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:48.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.221+0000 7f4a26ffd640 1 -- 172.21.15.69:0/3035496801 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a20039070 con 0x7f4a280b4570 2024-04-25T10:59:48.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.221+0000 7f4a26ffd640 1 -- 172.21.15.69:0/3035496801 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a20009600 con 0x7f4a280b4570 2024-04-25T10:59:48.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.221+0000 7f4a26ffd640 1 -- 172.21.15.69:0/3035496801 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a200099d0 con 0x7f4a280b4570 2024-04-25T10:59:48.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.221+0000 7f4a2e630640 1 -- 172.21.15.69:0/3035496801 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a280b4570 msgr2=0x7f4a280b4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:48.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.221+0000 7f4a2e630640 1 --2- 172.21.15.69:0/3035496801 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a280b4570 0x7f4a280b4980 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4a2000d150 tx=0x7f4a2002c890 comp rx=0 tx=0).stop 2024-04-25T10:59:48.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.221+0000 7f4a2e630640 1 -- 172.21.15.69:0/3035496801 shutdown_connections 2024-04-25T10:59:48.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.221+0000 7f4a2e630640 1 --2- 172.21.15.69:0/3035496801 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a280b4570 0x7f4a280b4980 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:48.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.221+0000 7f4a2e630640 1 --2- 172.21.15.69:0/3035496801 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a2806e0e0 0x7f4a280cff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:48.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.221+0000 7f4a2e630640 1 -- 172.21.15.69:0/3035496801 >> 172.21.15.69:0/3035496801 conn(0x7f4a2809b7d0 msgr2=0x7f4a2809bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:48.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.221+0000 7f4a2e630640 1 -- 172.21.15.69:0/3035496801 shutdown_connections 2024-04-25T10:59:48.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.221+0000 7f4a2e630640 1 -- 172.21.15.69:0/3035496801 wait complete. 2024-04-25T10:59:48.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.223+0000 7f4a2e630640 1 Processor -- start 2024-04-25T10:59:48.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.223+0000 7f4a2e630640 1 -- start start 2024-04-25T10:59:48.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.225+0000 7f4a2e630640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a2806e0e0 0x7f4a2815f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:48.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.225+0000 7f4a2e630640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a280b4570 0x7f4a280d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:48.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.225+0000 7f4a2e630640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a2815d130 0x7f4a280d2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:48.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.225+0000 7f4a2e630640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4a28160e40 con 0x7f4a280b4570 2024-04-25T10:59:48.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.225+0000 7f4a2e630640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2806ed80 con 0x7f4a2806e0e0 2024-04-25T10:59:48.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.225+0000 7f4a2e630640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4a2815d540 con 0x7f4a2815d130 2024-04-25T10:59:48.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.225+0000 7f4a277fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a280b4570 0x7f4a280d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:48.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.225+0000 7f4a277fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a280b4570 0x7f4a280d29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55052/0 (socket says 172.21.15.69:55052) 2024-04-25T10:59:48.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.225+0000 7f4a277fe640 1 -- 172.21.15.69:0/540977989 learned_addr learned my addr 172.21.15.69:0/540977989 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:48.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.225+0000 7f4a27fff640 1 --2- 172.21.15.69:0/540977989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a2806e0e0 0x7f4a2815f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:48.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.225+0000 7f4a2cbcd640 1 --2- 172.21.15.69:0/540977989 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a2815d130 0x7f4a280d2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:48.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.225+0000 7f4a277fe640 1 -- 172.21.15.69:0/540977989 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a2815d130 msgr2=0x7f4a280d2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:48.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.225+0000 7f4a277fe640 1 --2- 172.21.15.69:0/540977989 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a2815d130 0x7f4a280d2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:48.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.225+0000 7f4a277fe640 1 -- 172.21.15.69:0/540977989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a2806e0e0 msgr2=0x7f4a2815f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:48.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.225+0000 7f4a277fe640 1 --2- 172.21.15.69:0/540977989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a2806e0e0 0x7f4a2815f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:48.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.225+0000 7f4a277fe640 1 -- 172.21.15.69:0/540977989 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a280b2060 con 0x7f4a280b4570 2024-04-25T10:59:48.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.225+0000 7f4a277fe640 1 --2- 172.21.15.69:0/540977989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a280b4570 0x7f4a280d29f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c00c440 tx=0x7f4a1c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:48.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.225+0000 7f4a24ff9640 1 -- 172.21.15.69:0/540977989 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c018070 con 0x7f4a280b4570 2024-04-25T10:59:48.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.226+0000 7f4a24ff9640 1 -- 172.21.15.69:0/540977989 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a1c007500 con 0x7f4a280b4570 2024-04-25T10:59:48.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.226+0000 7f4a2e630640 1 -- 172.21.15.69:0/540977989 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a280b0fc0 con 0x7f4a280b4570 2024-04-25T10:59:48.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.226+0000 7f4a24ff9640 1 -- 172.21.15.69:0/540977989 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c004a20 con 0x7f4a280b4570 2024-04-25T10:59:48.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.226+0000 7f4a2e630640 1 -- 172.21.15.69:0/540977989 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a280b1580 con 0x7f4a280b4570 2024-04-25T10:59:48.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.226+0000 7f4a2e630640 1 -- 172.21.15.69:0/540977989 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a28050740 con 0x7f4a280b4570 2024-04-25T10:59:48.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.227+0000 7f4a24ff9640 1 -- 172.21.15.69:0/540977989 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4a1c01b070 con 0x7f4a280b4570 2024-04-25T10:59:48.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.228+0000 7f4a24ff9640 1 --2- 172.21.15.69:0/540977989 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4a140744e0 0x7f4a140769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:48.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.228+0000 7f4a24ff9640 1 -- 172.21.15.69:0/540977989 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4a1c0944c0 con 0x7f4a280b4570 2024-04-25T10:59:48.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.228+0000 7f4a27fff640 1 --2- 172.21.15.69:0/540977989 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4a140744e0 0x7f4a140769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:48.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.228+0000 7f4a27fff640 1 --2- 172.21.15.69:0/540977989 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4a140744e0 0x7f4a140769f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4a18009560 tx=0x7f4a18009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:48.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.232+0000 7f4a24ff9640 1 -- 172.21.15.69:0/540977989 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4a2806e4f0 con 0x7f4a280b4570 2024-04-25T10:59:48.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.405+0000 7f4a2e630640 1 -- 172.21.15.69:0/540977989 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a2806da20 con 0x7f4a280b4570 2024-04-25T10:59:48.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.406+0000 7f4a24ff9640 1 -- 172.21.15.69:0/540977989 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f4a1c05e600 con 0x7f4a280b4570 2024-04-25T10:59:48.407 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:48.407 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":20,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T10:59:20.630151+0000","last_up_change":"2024-04-25T10:59:05.546839+0000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:59:06.422674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:59:16.727980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:59:17.938486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6825","nonce":1876101065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6827","nonce":1876101065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6831","nonce":1876101065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6829","nonce":1876101065}]},"public_addr":"172.21.15.69:6825/1876101065","cluster_addr":"172.21.15.69:6827/1876101065","heartbeat_back_addr":"172.21.15.69:6831/1876101065","heartbeat_front_addr":"172.21.15.69:6829/1876101065","state":["exists","up"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6801","nonce":1806438563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6803","nonce":1806438563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6807","nonce":1806438563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6805","nonce":1806438563}]},"public_addr":"172.21.15.69:6801/1806438563","cluster_addr":"172.21.15.69:6803/1806438563","heartbeat_back_addr":"172.21.15.69:6807/1806438563","heartbeat_front_addr":"172.21.15.69:6805/1806438563","state":["exists","up"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6817","nonce":1593617983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6819","nonce":1593617983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6823","nonce":1593617983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6821","nonce":1593617983}]},"public_addr":"172.21.15.69:6817/1593617983","cluster_addr":"172.21.15.69:6819/1593617983","heartbeat_back_addr":"172.21.15.69:6823/1593617983","heartbeat_front_addr":"172.21.15.69:6821/1593617983","state":["exists","up"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6809","nonce":3950661294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6811","nonce":3950661294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6815","nonce":3950661294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6813","nonce":3950661294}]},"public_addr":"172.21.15.69:6809/3950661294","cluster_addr":"172.21.15.69:6811/3950661294","heartbeat_back_addr":"172.21.15.69:6815/3950661294","heartbeat_front_addr":"172.21.15.69:6813/3950661294","state":["exists","up"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6801","nonce":2508929872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6803","nonce":2508929872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6807","nonce":2508929872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6805","nonce":2508929872}]},"public_addr":"172.21.15.146:6801/2508929872","cluster_addr":"172.21.15.146:6803/2508929872","heartbeat_back_addr":"172.21.15.146:6807/2508929872","heartbeat_front_addr":"172.21.15.146:6805/2508929872","state":["exists","up"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6811","nonce":1310911138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6816","nonce":1310911138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6823","nonce":1310911138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6821","nonce":1310911138}]},"public_addr":"172.21.15.146:6811/1310911138","cluster_addr":"172.21.15.146:6816/1310911138","heartbeat_back_addr":"172.21.15.146:6823/1310911138","heartbeat_front_addr":"172.21.15.146:6821/1310911138","state":["exists","up"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6810","nonce":634688021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6813","nonce":634688021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6820","nonce":634688021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6817","nonce":634688021}]},"public_addr":"172.21.15.146:6810/634688021","cluster_addr":"172.21.15.146:6813/634688021","heartbeat_back_addr":"172.21.15.146:6820/634688021","heartbeat_front_addr":"172.21.15.146:6817/634688021","state":["exists","up"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6825","nonce":3814215935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6827","nonce":3814215935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6831","nonce":3814215935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6829","nonce":3814215935}]},"public_addr":"172.21.15.146:6825/3814215935","cluster_addr":"172.21.15.146:6827/3814215935","heartbeat_back_addr":"172.21.15.146:6831/3814215935","heartbeat_front_addr":"172.21.15.146:6829/3814215935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.820200+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.850189+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.764560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.866248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.807897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.869881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.782310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.827131+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:59:48.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.408+0000 7f4a2e630640 1 -- 172.21.15.69:0/540977989 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4a140744e0 msgr2=0x7f4a140769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:48.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.408+0000 7f4a2e630640 1 --2- 172.21.15.69:0/540977989 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4a140744e0 0x7f4a140769f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4a18009560 tx=0x7f4a18009190 comp rx=0 tx=0).stop 2024-04-25T10:59:48.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.408+0000 7f4a2e630640 1 -- 172.21.15.69:0/540977989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a280b4570 msgr2=0x7f4a280d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:48.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.408+0000 7f4a2e630640 1 --2- 172.21.15.69:0/540977989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a280b4570 0x7f4a280d29f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c00c440 tx=0x7f4a1c00c900 comp rx=0 tx=0).stop 2024-04-25T10:59:48.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.408+0000 7f4a2e630640 1 -- 172.21.15.69:0/540977989 shutdown_connections 2024-04-25T10:59:48.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.408+0000 7f4a2e630640 1 --2- 172.21.15.69:0/540977989 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4a140744e0 0x7f4a140769f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:48.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.408+0000 7f4a2e630640 1 --2- 172.21.15.69:0/540977989 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a2815d130 0x7f4a280d2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:48.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.408+0000 7f4a2e630640 1 --2- 172.21.15.69:0/540977989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a280b4570 0x7f4a280d29f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:48.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.408+0000 7f4a2e630640 1 --2- 172.21.15.69:0/540977989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a2806e0e0 0x7f4a2815f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:48.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.408+0000 7f4a2e630640 1 -- 172.21.15.69:0/540977989 >> 172.21.15.69:0/540977989 conn(0x7f4a2809b7d0 msgr2=0x7f4a2815ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:48.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.408+0000 7f4a2e630640 1 -- 172.21.15.69:0/540977989 shutdown_connections 2024-04-25T10:59:48.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.408+0000 7f4a2e630640 1 -- 172.21.15.69:0/540977989 wait complete. 2024-04-25T10:59:48.420 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:59:06.422674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:59:16.727980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:59:17.938486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:59:48.421 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:59:48.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.570+0000 7fb206538640 1 Processor -- start 2024-04-25T10:59:48.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.570+0000 7fb206538640 1 -- start start 2024-04-25T10:59:48.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.571+0000 7fb206538640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb20006ecd0 0x7fb2000b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:48.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.571+0000 7fb206538640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb200160640 0x7fb2000b7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:48.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.571+0000 7fb206538640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2000b3f30 con 0x7fb2000d0190 2024-04-25T10:59:48.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.571+0000 7fb206538640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fb2000b23e0 con 0x7fb20006ecd0 2024-04-25T10:59:48.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.571+0000 7fb206538640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fb2000b4210 con 0x7fb200160640 2024-04-25T10:59:48.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.571+0000 7fb1ff7fe640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb20006ecd0 0x7fb2000b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:48.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.572+0000 7fb1ff7fe640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb20006ecd0 0x7fb2000b77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55054/0 (socket says 172.21.15.69:55054) 2024-04-25T10:59:48.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.572+0000 7fb1ff7fe640 1 -- 172.21.15.69:0/4065608268 learned_addr learned my addr 172.21.15.69:0/4065608268 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:48.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.572+0000 7fb204ad5640 1 --2- 172.21.15.69:0/4065608268 >> v2:172.21.15.146:3300/0 conn(0x7fb200160640 0x7fb2000b7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:48.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.572+0000 7fb1ff7fe640 1 -- 172.21.15.69:0/4065608268 >> v2:172.21.15.146:3300/0 conn(0x7fb200160640 msgr2=0x7fb2000b7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:48.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.572+0000 7fb1ff7fe640 1 --2- 172.21.15.69:0/4065608268 >> v2:172.21.15.146:3300/0 conn(0x7fb200160640 0x7fb2000b7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:48.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.572+0000 7fb1ff7fe640 1 -- 172.21.15.69:0/4065608268 >> v1:172.21.15.146:6789/0 conn(0x7fb2000d0190 legacy=0x7fb2000d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:48.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.572+0000 7fb1ff7fe640 1 -- 172.21.15.69:0/4065608268 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2000b2630 con 0x7fb20006ecd0 2024-04-25T10:59:48.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.573+0000 7fb1ff7fe640 1 --2- 172.21.15.69:0/4065608268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb20006ecd0 0x7fb2000b77e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb2000b2560 tx=0x7fb1f402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15dc2c307c41ef98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:48.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.573+0000 7fb1feffd640 1 -- 172.21.15.69:0/4065608268 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4037070 con 0x7fb20006ecd0 2024-04-25T10:59:48.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.573+0000 7fb1feffd640 1 -- 172.21.15.69:0/4065608268 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1f4032440 con 0x7fb20006ecd0 2024-04-25T10:59:48.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.573+0000 7fb1feffd640 1 -- 172.21.15.69:0/4065608268 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4032810 con 0x7fb20006ecd0 2024-04-25T10:59:48.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.573+0000 7fb206538640 1 -- 172.21.15.69:0/4065608268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb20006ecd0 msgr2=0x7fb2000b77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:48.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.573+0000 7fb206538640 1 --2- 172.21.15.69:0/4065608268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb20006ecd0 0x7fb2000b77e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb2000b2560 tx=0x7fb1f402a140 comp rx=0 tx=0).stop 2024-04-25T10:59:48.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.573+0000 7fb206538640 1 -- 172.21.15.69:0/4065608268 shutdown_connections 2024-04-25T10:59:48.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.573+0000 7fb206538640 1 --2- 172.21.15.69:0/4065608268 >> v2:172.21.15.146:3300/0 conn(0x7fb200160640 0x7fb2000b7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:48.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.573+0000 7fb206538640 1 --2- 172.21.15.69:0/4065608268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb20006ecd0 0x7fb2000b77e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:48.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.573+0000 7fb206538640 1 -- 172.21.15.69:0/4065608268 >> 172.21.15.69:0/4065608268 conn(0x7fb20009b7d0 msgr2=0x7fb20009bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:48.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.573+0000 7fb206538640 1 -- 172.21.15.69:0/4065608268 shutdown_connections 2024-04-25T10:59:48.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.574+0000 7fb206538640 1 -- 172.21.15.69:0/4065608268 wait complete. 2024-04-25T10:59:48.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.575+0000 7fb206538640 1 Processor -- start 2024-04-25T10:59:48.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.575+0000 7fb206538640 1 -- start start 2024-04-25T10:59:48.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.577+0000 7fb206538640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb20006ecd0 0x7fb20014ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:48.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.577+0000 7fb206538640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2000d0190 0x7fb2000c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:48.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.577+0000 7fb206538640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb200160640 0x7fb2001fa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:48.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.577+0000 7fb206538640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb2000b3cd0 con 0x7fb20006ecd0 2024-04-25T10:59:48.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.577+0000 7fb206538640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb2000b3b50 con 0x7fb200160640 2024-04-25T10:59:48.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.577+0000 7fb206538640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb2000b2e30 con 0x7fb2000d0190 2024-04-25T10:59:48.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.577+0000 7fb1fffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb20006ecd0 0x7fb20014ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:48.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.577+0000 7fb1fffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb20006ecd0 0x7fb20014ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55068/0 (socket says 172.21.15.69:55068) 2024-04-25T10:59:48.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.577+0000 7fb1fffff640 1 -- 172.21.15.69:0/2038547080 learned_addr learned my addr 172.21.15.69:0/2038547080 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:48.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.577+0000 7fb204ad5640 1 --2- 172.21.15.69:0/2038547080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb200160640 0x7fb2001fa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:48.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.577+0000 7fb1fffff640 1 -- 172.21.15.69:0/2038547080 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2000d0190 msgr2=0x7fb2000c9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:48.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.577+0000 7fb1fffff640 1 --2- 172.21.15.69:0/2038547080 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2000d0190 0x7fb2000c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:48.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.577+0000 7fb1fffff640 1 -- 172.21.15.69:0/2038547080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb200160640 msgr2=0x7fb2001fa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:48.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.577+0000 7fb1fffff640 1 --2- 172.21.15.69:0/2038547080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb200160640 0x7fb2001fa2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:48.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.577+0000 7fb1fffff640 1 -- 172.21.15.69:0/2038547080 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2000ca910 con 0x7fb20006ecd0 2024-04-25T10:59:48.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.577+0000 7fb1fffff640 1 --2- 172.21.15.69:0/2038547080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb20006ecd0 0x7fb20014ce30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb1f000c470 tx=0x7fb1f000c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:48.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.577+0000 7fb1fcff9640 1 -- 172.21.15.69:0/2038547080 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f00052d0 con 0x7fb20006ecd0 2024-04-25T10:59:48.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.577+0000 7fb1fcff9640 1 -- 172.21.15.69:0/2038547080 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1f0005490 con 0x7fb20006ecd0 2024-04-25T10:59:48.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.578+0000 7fb206538640 1 -- 172.21.15.69:0/2038547080 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2000cabf0 con 0x7fb20006ecd0 2024-04-25T10:59:48.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.578+0000 7fb1fcff9640 1 -- 172.21.15.69:0/2038547080 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0007260 con 0x7fb20006ecd0 2024-04-25T10:59:48.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.578+0000 7fb206538640 1 -- 172.21.15.69:0/2038547080 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2000cb1b0 con 0x7fb20006ecd0 2024-04-25T10:59:48.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.578+0000 7fb206538640 1 -- 172.21.15.69:0/2038547080 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb200074ac0 con 0x7fb20006ecd0 2024-04-25T10:59:48.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.579+0000 7fb1fcff9640 1 -- 172.21.15.69:0/2038547080 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb1f0007400 con 0x7fb20006ecd0 2024-04-25T10:59:48.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.579+0000 7fb1fcff9640 1 --2- 172.21.15.69:0/2038547080 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb1ec0744b0 0x7fb1ec0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:48.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.580+0000 7fb1fcff9640 1 -- 172.21.15.69:0/2038547080 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb1f0094590 con 0x7fb20006ecd0 2024-04-25T10:59:48.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.580+0000 7fb1ff7fe640 1 --2- 172.21.15.69:0/2038547080 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb1ec0744b0 0x7fb1ec0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:48.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.580+0000 7fb1ff7fe640 1 --2- 172.21.15.69:0/2038547080 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb1ec0744b0 0x7fb1ec0769c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4034000 tx=0x7fb1f402c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:48.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.584+0000 7fb1fcff9640 1 -- 172.21.15.69:0/2038547080 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb200074ac0 con 0x7fb20006ecd0 2024-04-25T10:59:48.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.757+0000 7fb206538640 1 -- 172.21.15.69:0/2038547080 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb2000d1390 con 0x7fb20006ecd0 2024-04-25T10:59:48.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.758+0000 7fb1fcff9640 1 -- 172.21.15.69:0/2038547080 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fb2000d1390 con 0x7fb20006ecd0 2024-04-25T10:59:48.759 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-25T10:59:48.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.760+0000 7fb206538640 1 -- 172.21.15.69:0/2038547080 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb1ec0744b0 msgr2=0x7fb1ec0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:48.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.760+0000 7fb206538640 1 --2- 172.21.15.69:0/2038547080 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb1ec0744b0 0x7fb1ec0769c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4034000 tx=0x7fb1f402c000 comp rx=0 tx=0).stop 2024-04-25T10:59:48.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.760+0000 7fb206538640 1 -- 172.21.15.69:0/2038547080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb20006ecd0 msgr2=0x7fb20014ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:48.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.760+0000 7fb206538640 1 --2- 172.21.15.69:0/2038547080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb20006ecd0 0x7fb20014ce30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb1f000c470 tx=0x7fb1f000c930 comp rx=0 tx=0).stop 2024-04-25T10:59:48.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.760+0000 7fb206538640 1 -- 172.21.15.69:0/2038547080 shutdown_connections 2024-04-25T10:59:48.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.760+0000 7fb206538640 1 --2- 172.21.15.69:0/2038547080 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb1ec0744b0 0x7fb1ec0769c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:48.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.760+0000 7fb206538640 1 --2- 172.21.15.69:0/2038547080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb200160640 0x7fb2001fa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:48.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.760+0000 7fb206538640 1 --2- 172.21.15.69:0/2038547080 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2000d0190 0x7fb2000c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:48.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.760+0000 7fb206538640 1 --2- 172.21.15.69:0/2038547080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb20006ecd0 0x7fb20014ce30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:48.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.760+0000 7fb206538640 1 -- 172.21.15.69:0/2038547080 >> 172.21.15.69:0/2038547080 conn(0x7fb20009b7d0 msgr2=0x7fb2000cf780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:48.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.760+0000 7fb206538640 1 -- 172.21.15.69:0/2038547080 shutdown_connections 2024-04-25T10:59:48.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.760+0000 7fb206538640 1 -- 172.21.15.69:0/2038547080 wait complete. 2024-04-25T10:59:48.772 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:59:48.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.881+0000 7f3e633ce640 1 Processor -- start 2024-04-25T10:59:48.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.881+0000 7f3e633ce640 1 -- start start 2024-04-25T10:59:48.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.882+0000 7f3e633ce640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e5c0b0c90 0x7f3e5c0704c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:48.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.883+0000 7f3e633ce640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3e5c15adf0 0x7f3e5c0cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:48.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.883+0000 7f3e633ce640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f3e5c06d5d0 0x7f3e5c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:48.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.883+0000 7f3e633ce640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f3e5c06df20 con 0x7f3e5c06d5d0 2024-04-25T10:59:48.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.883+0000 7f3e633ce640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f3e5c06fa20 con 0x7f3e5c15adf0 2024-04-25T10:59:48.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.883+0000 7f3e633ce640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3e5c070fd0 con 0x7f3e5c0b0c90 2024-04-25T10:59:48.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.883+0000 7f3e6196b640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f3e5c06d5d0 0x7f3e5c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:48.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.883+0000 7f3e6196b640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f3e5c06d5d0 0x7f3e5c06d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55072/0 (socket says 172.21.15.69:55072) 2024-04-25T10:59:48.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.883+0000 7f3e6196b640 1 -- 172.21.15.69:0/4044440595 learned_addr learned my addr 172.21.15.69:0/4044440595 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:48.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.883+0000 7f3e6196b640 1 -- 172.21.15.69:0/4044440595 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e5c0b0c90 msgr2=0x7f3e5c0704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:48.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.883+0000 7f3e6196b640 1 --2- 172.21.15.69:0/4044440595 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e5c0b0c90 0x7f3e5c0704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:48.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.883+0000 7f3e6196b640 1 -- 172.21.15.69:0/4044440595 >> v2:172.21.15.146:3300/0 conn(0x7f3e5c15adf0 msgr2=0x7f3e5c0cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:48.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.883+0000 7f3e6196b640 1 --2- 172.21.15.69:0/4044440595 >> v2:172.21.15.146:3300/0 conn(0x7f3e5c15adf0 0x7f3e5c0cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:48.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.883+0000 7f3e6196b640 1 -- 172.21.15.69:0/4044440595 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e5c0712b0 con 0x7f3e5c06d5d0 2024-04-25T10:59:48.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.884+0000 7f3e6196b640 1 --2- 172.21.15.69:0/4044440595 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e5c06d5d0 0x7f3e5c06d9e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c0b4530 tx=0x7f3e5402c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19de14991ebcdd0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:48.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.884+0000 7f3e53fff640 1 -- 172.21.15.69:0/4044440595 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e54033640 con 0x7f3e5c06d5d0 2024-04-25T10:59:48.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.884+0000 7f3e53fff640 1 -- 172.21.15.69:0/4044440595 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e54038070 con 0x7f3e5c06d5d0 2024-04-25T10:59:48.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.884+0000 7f3e53fff640 1 -- 172.21.15.69:0/4044440595 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e54033a00 con 0x7f3e5c06d5d0 2024-04-25T10:59:48.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.884+0000 7f3e633ce640 1 -- 172.21.15.69:0/4044440595 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e5c06d5d0 msgr2=0x7f3e5c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:48.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.884+0000 7f3e633ce640 1 --2- 172.21.15.69:0/4044440595 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e5c06d5d0 0x7f3e5c06d9e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c0b4530 tx=0x7f3e5402c8b0 comp rx=0 tx=0).stop 2024-04-25T10:59:48.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.885+0000 7f3e633ce640 1 -- 172.21.15.69:0/4044440595 shutdown_connections 2024-04-25T10:59:48.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.885+0000 7f3e633ce640 1 --2- 172.21.15.69:0/4044440595 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e5c06d5d0 0x7f3e5c06d9e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:48.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.885+0000 7f3e633ce640 1 --2- 172.21.15.69:0/4044440595 >> v2:172.21.15.146:3300/0 conn(0x7f3e5c15adf0 0x7f3e5c0cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:48.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.885+0000 7f3e633ce640 1 --2- 172.21.15.69:0/4044440595 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e5c0b0c90 0x7f3e5c0704c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:48.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.885+0000 7f3e633ce640 1 -- 172.21.15.69:0/4044440595 >> 172.21.15.69:0/4044440595 conn(0x7f3e5c073090 msgr2=0x7f3e5c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:48.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.885+0000 7f3e633ce640 1 -- 172.21.15.69:0/4044440595 shutdown_connections 2024-04-25T10:59:48.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.885+0000 7f3e633ce640 1 -- 172.21.15.69:0/4044440595 wait complete. 2024-04-25T10:59:48.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.886+0000 7f3e633ce640 1 Processor -- start 2024-04-25T10:59:48.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.886+0000 7f3e633ce640 1 -- start start 2024-04-25T10:59:48.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.888+0000 7f3e633ce640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e5c06d5d0 0x7f3e5c0b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:48.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.888+0000 7f3e633ce640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e5c0b0c90 0x7f3e5c0b1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:48.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.888+0000 7f3e633ce640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e5c15adf0 0x7f3e5c0ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:48.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.888+0000 7f3e633ce640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5c070c80 con 0x7f3e5c06d5d0 2024-04-25T10:59:48.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.888+0000 7f3e633ce640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5c06ffa0 con 0x7f3e5c0b0c90 2024-04-25T10:59:48.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.888+0000 7f3e633ce640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3e5c15d790 con 0x7f3e5c15adf0 2024-04-25T10:59:48.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.888+0000 7f3e6116a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e5c06d5d0 0x7f3e5c0b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:48.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.888+0000 7f3e6116a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e5c06d5d0 0x7f3e5c0b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55074/0 (socket says 172.21.15.69:55074) 2024-04-25T10:59:48.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.888+0000 7f3e6116a640 1 -- 172.21.15.69:0/1849467909 learned_addr learned my addr 172.21.15.69:0/1849467909 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:48.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.888+0000 7f3e60969640 1 --2- 172.21.15.69:0/1849467909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e5c0b0c90 0x7f3e5c0b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:48.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.888+0000 7f3e6196b640 1 --2- 172.21.15.69:0/1849467909 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e5c15adf0 0x7f3e5c0ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:48.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.888+0000 7f3e6116a640 1 -- 172.21.15.69:0/1849467909 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e5c15adf0 msgr2=0x7f3e5c0ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:48.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.888+0000 7f3e6116a640 1 --2- 172.21.15.69:0/1849467909 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e5c15adf0 0x7f3e5c0ae640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:48.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.888+0000 7f3e6116a640 1 -- 172.21.15.69:0/1849467909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e5c0b0c90 msgr2=0x7f3e5c0b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:48.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.888+0000 7f3e6116a640 1 --2- 172.21.15.69:0/1849467909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e5c0b0c90 0x7f3e5c0b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:48.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.888+0000 7f3e6116a640 1 -- 172.21.15.69:0/1849467909 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e5c0ce240 con 0x7f3e5c06d5d0 2024-04-25T10:59:48.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.888+0000 7f3e60969640 1 --2- 172.21.15.69:0/1849467909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e5c0b0c90 0x7f3e5c0b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:48.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.888+0000 7f3e6116a640 1 --2- 172.21.15.69:0/1849467909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e5c06d5d0 0x7f3e5c0b12e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3e5800b9c0 tx=0x7f3e5800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:48.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.888+0000 7f3e51ffb640 1 -- 172.21.15.69:0/1849467909 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e58019070 con 0x7f3e5c06d5d0 2024-04-25T10:59:48.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.888+0000 7f3e51ffb640 1 -- 172.21.15.69:0/1849467909 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e5800cba0 con 0x7f3e5c06d5d0 2024-04-25T10:59:48.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.888+0000 7f3e633ce640 1 -- 172.21.15.69:0/1849467909 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e5c1569c0 con 0x7f3e5c06d5d0 2024-04-25T10:59:48.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.888+0000 7f3e51ffb640 1 -- 172.21.15.69:0/1849467909 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e58014440 con 0x7f3e5c06d5d0 2024-04-25T10:59:48.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.889+0000 7f3e633ce640 1 -- 172.21.15.69:0/1849467909 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e5c155090 con 0x7f3e5c06d5d0 2024-04-25T10:59:48.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.890+0000 7f3e633ce640 1 -- 172.21.15.69:0/1849467909 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e5c156ee0 con 0x7f3e5c06d5d0 2024-04-25T10:59:48.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.890+0000 7f3e51ffb640 1 -- 172.21.15.69:0/1849467909 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3e5801c070 con 0x7f3e5c06d5d0 2024-04-25T10:59:48.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.890+0000 7f3e51ffb640 1 --2- 172.21.15.69:0/1849467909 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3e400744b0 0x7f3e400769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:48.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.890+0000 7f3e51ffb640 1 -- 172.21.15.69:0/1849467909 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3e58094ea0 con 0x7f3e5c06d5d0 2024-04-25T10:59:48.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.890+0000 7f3e60969640 1 --2- 172.21.15.69:0/1849467909 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3e400744b0 0x7f3e400769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:48.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.891+0000 7f3e60969640 1 --2- 172.21.15.69:0/1849467909 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3e400744b0 0x7f3e400769c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c0ce540 tx=0x7f3e4c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:48.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:48.895+0000 7f3e51ffb640 1 -- 172.21.15.69:0/1849467909 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3e5c156ee0 con 0x7f3e5c06d5d0 2024-04-25T10:59:49.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.068+0000 7f3e633ce640 1 -- 172.21.15.69:0/1849467909 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3e5c050740 con 0x7f3e5c06d5d0 2024-04-25T10:59:49.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.069+0000 7f3e51ffb640 1 -- 172.21.15.69:0/1849467909 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f3e5805f750 con 0x7f3e5c06d5d0 2024-04-25T10:59:49.070 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-25T10:59:49.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.071+0000 7f3e633ce640 1 -- 172.21.15.69:0/1849467909 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3e400744b0 msgr2=0x7f3e400769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:49.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.071+0000 7f3e633ce640 1 --2- 172.21.15.69:0/1849467909 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3e400744b0 0x7f3e400769c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c0ce540 tx=0x7f3e4c008040 comp rx=0 tx=0).stop 2024-04-25T10:59:49.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.071+0000 7f3e633ce640 1 -- 172.21.15.69:0/1849467909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e5c06d5d0 msgr2=0x7f3e5c0b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:49.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.071+0000 7f3e633ce640 1 --2- 172.21.15.69:0/1849467909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e5c06d5d0 0x7f3e5c0b12e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3e5800b9c0 tx=0x7f3e5800be80 comp rx=0 tx=0).stop 2024-04-25T10:59:49.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.071+0000 7f3e633ce640 1 -- 172.21.15.69:0/1849467909 shutdown_connections 2024-04-25T10:59:49.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.071+0000 7f3e633ce640 1 --2- 172.21.15.69:0/1849467909 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3e400744b0 0x7f3e400769c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:49.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.071+0000 7f3e633ce640 1 --2- 172.21.15.69:0/1849467909 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e5c15adf0 0x7f3e5c0ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:49.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.071+0000 7f3e633ce640 1 --2- 172.21.15.69:0/1849467909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e5c0b0c90 0x7f3e5c0b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:49.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.071+0000 7f3e633ce640 1 --2- 172.21.15.69:0/1849467909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e5c06d5d0 0x7f3e5c0b12e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:49.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.071+0000 7f3e633ce640 1 -- 172.21.15.69:0/1849467909 >> 172.21.15.69:0/1849467909 conn(0x7f3e5c073090 msgr2=0x7f3e5c0d0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:49.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.071+0000 7f3e633ce640 1 -- 172.21.15.69:0/1849467909 shutdown_connections 2024-04-25T10:59:49.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.071+0000 7f3e633ce640 1 -- 172.21.15.69:0/1849467909 wait complete. 2024-04-25T10:59:49.083 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:59:49.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.192+0000 7f3d35669640 1 Processor -- start 2024-04-25T10:59:49.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.192+0000 7f3d35669640 1 -- start start 2024-04-25T10:59:49.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.194+0000 7f3d35669640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3d3015e080 0x7f3d301646b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:49.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.194+0000 7f3d35669640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d301609b0 con 0x7f3d30161710 2024-04-25T10:59:49.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.194+0000 7f3d35669640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d30164cd0 con 0x7f3d3006ccd0 2024-04-25T10:59:49.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.194+0000 7f3d35669640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f3d30160c30 con 0x7f3d3015e080 2024-04-25T10:59:49.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.194+0000 7f3d2effd640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f3d30161710 0x7f3d3006c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:50156/0 (socket says 172.21.15.69:50156) 2024-04-25T10:59:49.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.194+0000 7f3d2effd640 1 -- 172.21.15.69:0/1481901584 learned_addr learned my addr 172.21.15.69:0/1481901584 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:49.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.194+0000 7f3d2dffb640 1 -- 172.21.15.69:0/1481901584 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4073840184 0 0) 0x7f3d20001490 con 0x7f3d30161710 2024-04-25T10:59:49.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.194+0000 7f3d2f7fe640 1 --2- 172.21.15.69:0/1481901584 >> v2:172.21.15.146:3300/0 conn(0x7f3d3015e080 0x7f3d301646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:49.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.194+0000 7f3d2dffb640 1 -- 172.21.15.69:0/1481901584 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d18003520 con 0x7f3d30161710 2024-04-25T10:59:49.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.194+0000 7f3d2dffb640 1 -- 172.21.15.69:0/1481901584 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 318016317 0 0) 0x7f3d18003520 con 0x7f3d30161710 2024-04-25T10:59:49.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.194+0000 7f3d2dffb640 1 -- 172.21.15.69:0/1481901584 >> v2:172.21.15.146:3300/0 conn(0x7f3d3015e080 msgr2=0x7f3d301646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:49.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.194+0000 7f3d2dffb640 1 --2- 172.21.15.69:0/1481901584 >> v2:172.21.15.146:3300/0 conn(0x7f3d3015e080 0x7f3d301646b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:49.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.194+0000 7f3d2dffb640 1 -- 172.21.15.69:0/1481901584 >> v1:172.21.15.146:6789/0 conn(0x7f3d3006ccd0 legacy=0x7f3d30163fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:49.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.194+0000 7f3d2dffb640 1 -- 172.21.15.69:0/1481901584 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d30160f10 con 0x7f3d30161710 2024-04-25T10:59:49.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.194+0000 7f3d2dffb640 1 -- 172.21.15.69:0/1481901584 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7f3d20001bc0 con 0x7f3d30161710 2024-04-25T10:59:49.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.194+0000 7f3d2dffb640 1 -- 172.21.15.69:0/1481901584 >> v1:172.21.15.69:6789/0 conn(0x7f3d30161710 legacy=0x7f3d3006c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:49.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.194+0000 7f3d2dffb640 1 --2- 172.21.15.69:0/1481901584 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d180039b0 0x7f3d18003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:49.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.194+0000 7f3d2dffb640 1 --2- 172.21.15.69:0/1481901584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d18004800 0x7f3d18004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:49.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.195+0000 7f3d2dffb640 1 --2- 172.21.15.69:0/1481901584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d18005250 0x7f3d1800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:49.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.195+0000 7f3d2dffb640 1 -- 172.21.15.69:0/1481901584 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3d30160c30 con 0x7f3d18004800 2024-04-25T10:59:49.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.195+0000 7f3d2dffb640 1 -- 172.21.15.69:0/1481901584 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1800c1e0 con 0x7f3d18005250 2024-04-25T10:59:49.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.195+0000 7f3d2dffb640 1 -- 172.21.15.69:0/1481901584 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3d1800c3e0 con 0x7f3d180039b0 2024-04-25T10:59:49.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.195+0000 7f3d2e7fc640 1 --2- 172.21.15.69:0/1481901584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d18004800 0x7f3d18004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:49.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.195+0000 7f3d2f7fe640 1 --2- 172.21.15.69:0/1481901584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d18005250 0x7f3d1800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:49.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.195+0000 7f3d2effd640 1 --2- 172.21.15.69:0/1481901584 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d180039b0 0x7f3d18003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:49.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.195+0000 7f3d2e7fc640 1 -- 172.21.15.69:0/1481901584 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d180039b0 msgr2=0x7f3d18003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:49.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.195+0000 7f3d2e7fc640 1 --2- 172.21.15.69:0/1481901584 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d180039b0 0x7f3d18003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:49.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.195+0000 7f3d2e7fc640 1 -- 172.21.15.69:0/1481901584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d18005250 msgr2=0x7f3d1800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:49.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.195+0000 7f3d2e7fc640 1 --2- 172.21.15.69:0/1481901584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d18005250 0x7f3d1800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:49.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.195+0000 7f3d2e7fc640 1 -- 172.21.15.69:0/1481901584 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d18003520 con 0x7f3d18004800 2024-04-25T10:59:49.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.195+0000 7f3d2f7fe640 1 -- 172.21.15.69:0/1481901584 reap_dead start 2024-04-25T10:59:49.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.196+0000 7f3d2e7fc640 1 --2- 172.21.15.69:0/1481901584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d18004800 0x7f3d18004c10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3d24001750 tx=0x7f3d18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e61300a83afb089 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:49.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.196+0000 7f3d2dffb640 1 -- 172.21.15.69:0/1481901584 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24001370 con 0x7f3d18004800 2024-04-25T10:59:49.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.196+0000 7f3d2dffb640 1 -- 172.21.15.69:0/1481901584 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d240052d0 con 0x7f3d18004800 2024-04-25T10:59:49.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.196+0000 7f3d2dffb640 1 -- 172.21.15.69:0/1481901584 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2402abc0 con 0x7f3d18004800 2024-04-25T10:59:49.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.196+0000 7f3d35669640 1 -- 172.21.15.69:0/1481901584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d18004800 msgr2=0x7f3d18004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:49.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.196+0000 7f3d35669640 1 --2- 172.21.15.69:0/1481901584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d18004800 0x7f3d18004c10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3d24001750 tx=0x7f3d18005830 comp rx=0 tx=0).stop 2024-04-25T10:59:49.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.196+0000 7f3d35669640 1 -- 172.21.15.69:0/1481901584 shutdown_connections 2024-04-25T10:59:49.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.196+0000 7f3d35669640 1 --2- 172.21.15.69:0/1481901584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d18004800 0x7f3d18004c10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:49.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.196+0000 7f3d35669640 1 -- 172.21.15.69:0/1481901584 >> 172.21.15.69:0/1481901584 conn(0x7f3d300d5d90 msgr2=0x7f3d300d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:49.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.197+0000 7f3d35669640 1 -- 172.21.15.69:0/1481901584 shutdown_connections 2024-04-25T10:59:49.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.197+0000 7f3d35669640 1 -- 172.21.15.69:0/1481901584 wait complete. 2024-04-25T10:59:49.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.198+0000 7f3d35669640 1 Processor -- start 2024-04-25T10:59:49.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.198+0000 7f3d35669640 1 -- start start 2024-04-25T10:59:49.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.200+0000 7f3d35669640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d18004800 0x7f3d300b82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:49.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.200+0000 7f3d35669640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d3014f210 0x7f3d300cd750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:49.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.200+0000 7f3d35669640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d300cdc90 0x7f3d300baea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:49.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.200+0000 7f3d35669640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3006c740 con 0x7f3d3014f210 2024-04-25T10:59:49.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.200+0000 7f3d35669640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3006c5c0 con 0x7f3d18004800 2024-04-25T10:59:49.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.200+0000 7f3d35669640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3d3006c8c0 con 0x7f3d300cdc90 2024-04-25T10:59:49.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.200+0000 7f3d2e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d3014f210 0x7f3d300cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:49.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.200+0000 7f3d2e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d3014f210 0x7f3d300cd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55080/0 (socket says 172.21.15.69:55080) 2024-04-25T10:59:49.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.200+0000 7f3d2e7fc640 1 -- 172.21.15.69:0/2708353957 learned_addr learned my addr 172.21.15.69:0/2708353957 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:49.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.200+0000 7f3d2effd640 1 --2- 172.21.15.69:0/2708353957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d18004800 0x7f3d300b82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:49.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.200+0000 7f3d2f7fe640 1 --2- 172.21.15.69:0/2708353957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d300cdc90 0x7f3d300baea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:49.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.200+0000 7f3d2e7fc640 1 -- 172.21.15.69:0/2708353957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d300cdc90 msgr2=0x7f3d300baea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:49.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.200+0000 7f3d2e7fc640 1 --2- 172.21.15.69:0/2708353957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d300cdc90 0x7f3d300baea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:49.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.200+0000 7f3d2e7fc640 1 -- 172.21.15.69:0/2708353957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d18004800 msgr2=0x7f3d300b82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:49.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.200+0000 7f3d2e7fc640 1 --2- 172.21.15.69:0/2708353957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d18004800 0x7f3d300b82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:49.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.200+0000 7f3d2e7fc640 1 -- 172.21.15.69:0/2708353957 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d300b6d10 con 0x7f3d3014f210 2024-04-25T10:59:49.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.200+0000 7f3d2e7fc640 1 --2- 172.21.15.69:0/2708353957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d3014f210 0x7f3d300cd750 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3d24000c00 tx=0x7f3d24001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:49.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.200+0000 7f3d177fe640 1 -- 172.21.15.69:0/2708353957 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2403f030 con 0x7f3d3014f210 2024-04-25T10:59:49.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.200+0000 7f3d177fe640 1 -- 172.21.15.69:0/2708353957 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d24005480 con 0x7f3d3014f210 2024-04-25T10:59:49.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.200+0000 7f3d35669640 1 -- 172.21.15.69:0/2708353957 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d300b7210 con 0x7f3d3014f210 2024-04-25T10:59:49.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.200+0000 7f3d177fe640 1 -- 172.21.15.69:0/2708353957 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24037040 con 0x7f3d3014f210 2024-04-25T10:59:49.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.201+0000 7f3d35669640 1 -- 172.21.15.69:0/2708353957 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d300b77d0 con 0x7f3d3014f210 2024-04-25T10:59:49.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.202+0000 7f3d35669640 1 -- 172.21.15.69:0/2708353957 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d30050740 con 0x7f3d3014f210 2024-04-25T10:59:49.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.202+0000 7f3d177fe640 1 -- 172.21.15.69:0/2708353957 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3d24042070 con 0x7f3d3014f210 2024-04-25T10:59:49.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.203+0000 7f3d177fe640 1 --2- 172.21.15.69:0/2708353957 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3d1c0744e0 0x7f3d1c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:49.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.203+0000 7f3d177fe640 1 -- 172.21.15.69:0/2708353957 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3d240b49a0 con 0x7f3d3014f210 2024-04-25T10:59:49.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.203+0000 7f3d2effd640 1 --2- 172.21.15.69:0/2708353957 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3d1c0744e0 0x7f3d1c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:49.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.203+0000 7f3d2effd640 1 --2- 172.21.15.69:0/2708353957 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3d1c0744e0 0x7f3d1c0769f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3d200013d0 tx=0x7f3d20001c60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:49.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.208+0000 7f3d177fe640 1 -- 172.21.15.69:0/2708353957 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3d24049070 con 0x7f3d3014f210 2024-04-25T10:59:49.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.382+0000 7f3d35669640 1 -- 172.21.15.69:0/2708353957 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3d30161bf0 con 0x7f3d3014f210 2024-04-25T10:59:49.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.382+0000 7f3d177fe640 1 -- 172.21.15.69:0/2708353957 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f3d2407f1c0 con 0x7f3d3014f210 2024-04-25T10:59:49.383 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-25T10:59:49.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.384+0000 7f3d35669640 1 -- 172.21.15.69:0/2708353957 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3d1c0744e0 msgr2=0x7f3d1c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:49.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.384+0000 7f3d35669640 1 --2- 172.21.15.69:0/2708353957 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3d1c0744e0 0x7f3d1c0769f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3d200013d0 tx=0x7f3d20001c60 comp rx=0 tx=0).stop 2024-04-25T10:59:49.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.384+0000 7f3d35669640 1 -- 172.21.15.69:0/2708353957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d3014f210 msgr2=0x7f3d300cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:49.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.384+0000 7f3d35669640 1 --2- 172.21.15.69:0/2708353957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d3014f210 0x7f3d300cd750 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3d24000c00 tx=0x7f3d24001120 comp rx=0 tx=0).stop 2024-04-25T10:59:49.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.384+0000 7f3d35669640 1 -- 172.21.15.69:0/2708353957 shutdown_connections 2024-04-25T10:59:49.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.384+0000 7f3d35669640 1 --2- 172.21.15.69:0/2708353957 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3d1c0744e0 0x7f3d1c0769f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:49.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.384+0000 7f3d35669640 1 --2- 172.21.15.69:0/2708353957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d300cdc90 0x7f3d300baea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:49.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.384+0000 7f3d35669640 1 --2- 172.21.15.69:0/2708353957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d3014f210 0x7f3d300cd750 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:49.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.384+0000 7f3d35669640 1 --2- 172.21.15.69:0/2708353957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d18004800 0x7f3d300b82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:49.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.384+0000 7f3d35669640 1 -- 172.21.15.69:0/2708353957 >> 172.21.15.69:0/2708353957 conn(0x7f3d300d5d90 msgr2=0x7f3d3015d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:49.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.384+0000 7f3d35669640 1 -- 172.21.15.69:0/2708353957 shutdown_connections 2024-04-25T10:59:49.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.384+0000 7f3d35669640 1 -- 172.21.15.69:0/2708353957 wait complete. 2024-04-25T10:59:49.396 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:59:49.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.503+0000 7f7018d5e640 1 Processor -- start 2024-04-25T10:59:49.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.503+0000 7f7018d5e640 1 -- start start 2024-04-25T10:59:49.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.505+0000 7f7018d5e640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f70140d4130 0x7f70140d1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:49.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.505+0000 7f7018d5e640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70140b3c10 con 0x7f701406d8e0 2024-04-25T10:59:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.505+0000 7f7018d5e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70140b3ef0 con 0x7f701406dcf0 2024-04-25T10:59:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.505+0000 7f7018d5e640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f70140b9520 con 0x7f70140d4130 2024-04-25T10:59:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.505+0000 7f701259c640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f701406d8e0 0x7f70140d3a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:50158/0 (socket says 172.21.15.69:50158) 2024-04-25T10:59:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.505+0000 7f701259c640 1 -- 172.21.15.69:0/2600765669 learned_addr learned my addr 172.21.15.69:0/2600765669 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.505+0000 7f7011d9b640 1 --2- 172.21.15.69:0/2600765669 >> v2:172.21.15.146:3300/0 conn(0x7f70140d4130 0x7f70140d1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.505+0000 7f701159a640 1 -- 172.21.15.69:0/2600765669 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2264627844 0 0) 0x7f70140b3c10 con 0x7f701406d8e0 2024-04-25T10:59:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.505+0000 7f701159a640 1 -- 172.21.15.69:0/2600765669 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ffc003520 con 0x7f701406d8e0 2024-04-25T10:59:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.505+0000 7f701159a640 1 -- 172.21.15.69:0/2600765669 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2017726492 0 0) 0x7f6ffc003520 con 0x7f701406d8e0 2024-04-25T10:59:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.505+0000 7f701159a640 1 -- 172.21.15.69:0/2600765669 >> v2:172.21.15.146:3300/0 conn(0x7f70140d4130 msgr2=0x7f70140d1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.505+0000 7f701159a640 1 --2- 172.21.15.69:0/2600765669 >> v2:172.21.15.146:3300/0 conn(0x7f70140d4130 0x7f70140d1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.505+0000 7f701159a640 1 -- 172.21.15.69:0/2600765669 >> v1:172.21.15.146:6789/0 conn(0x7f701406dcf0 legacy=0x7f70140d22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.505+0000 7f701159a640 1 -- 172.21.15.69:0/2600765669 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70140b41d0 con 0x7f701406d8e0 2024-04-25T10:59:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.505+0000 7f701159a640 1 -- 172.21.15.69:0/2600765669 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7f70040019f0 con 0x7f701406d8e0 2024-04-25T10:59:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.505+0000 7f701159a640 1 -- 172.21.15.69:0/2600765669 >> v1:172.21.15.69:6789/0 conn(0x7f701406d8e0 legacy=0x7f70140d3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:49.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.505+0000 7f701159a640 1 --2- 172.21.15.69:0/2600765669 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ffc0039b0 0x7f6ffc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:49.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.505+0000 7f701159a640 1 --2- 172.21.15.69:0/2600765669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ffc004800 0x7f6ffc004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:49.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.505+0000 7f701159a640 1 --2- 172.21.15.69:0/2600765669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ffc005250 0x7f6ffc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:49.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.505+0000 7f701159a640 1 -- 172.21.15.69:0/2600765669 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f70140b9520 con 0x7f6ffc005250 2024-04-25T10:59:49.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.505+0000 7f701159a640 1 -- 172.21.15.69:0/2600765669 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc00c1e0 con 0x7f6ffc004800 2024-04-25T10:59:49.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.505+0000 7f701159a640 1 -- 172.21.15.69:0/2600765669 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f6ffc00c3e0 con 0x7f6ffc0039b0 2024-04-25T10:59:49.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.506+0000 7f7012d9d640 1 --2- 172.21.15.69:0/2600765669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ffc005250 0x7f6ffc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:49.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.506+0000 7f7011d9b640 1 --2- 172.21.15.69:0/2600765669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ffc004800 0x7f6ffc004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:49.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.506+0000 7f701259c640 1 --2- 172.21.15.69:0/2600765669 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ffc0039b0 0x7f6ffc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:49.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.506+0000 7f7012d9d640 1 -- 172.21.15.69:0/2600765669 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ffc0039b0 msgr2=0x7f6ffc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:49.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.506+0000 7f7012d9d640 1 --2- 172.21.15.69:0/2600765669 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ffc0039b0 0x7f6ffc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:49.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.506+0000 7f7012d9d640 1 -- 172.21.15.69:0/2600765669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ffc004800 msgr2=0x7f6ffc004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:49.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.506+0000 7f7012d9d640 1 --2- 172.21.15.69:0/2600765669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ffc004800 0x7f6ffc004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:49.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.506+0000 7f7012d9d640 1 -- 172.21.15.69:0/2600765669 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ffc003520 con 0x7f6ffc005250 2024-04-25T10:59:49.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.506+0000 7f7012d9d640 1 -- 172.21.15.69:0/2600765669 reap_dead start 2024-04-25T10:59:49.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.507+0000 7f7012d9d640 1 --2- 172.21.15.69:0/2600765669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ffc005250 0x7f6ffc00b9c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f700c001720 tx=0x7f6ffc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53fa7bc573e6709 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:49.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.507+0000 7f701159a640 1 -- 172.21.15.69:0/2600765669 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c001370 con 0x7f6ffc005250 2024-04-25T10:59:49.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.507+0000 7f701159a640 1 -- 172.21.15.69:0/2600765669 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f700c0052d0 con 0x7f6ffc005250 2024-04-25T10:59:49.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.507+0000 7f701159a640 1 -- 172.21.15.69:0/2600765669 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c02abc0 con 0x7f6ffc005250 2024-04-25T10:59:49.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.507+0000 7f7018d5e640 1 -- 172.21.15.69:0/2600765669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ffc005250 msgr2=0x7f6ffc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:49.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.507+0000 7f7018d5e640 1 --2- 172.21.15.69:0/2600765669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ffc005250 0x7f6ffc00b9c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f700c001720 tx=0x7f6ffc005830 comp rx=0 tx=0).stop 2024-04-25T10:59:49.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.508+0000 7f7018d5e640 1 -- 172.21.15.69:0/2600765669 shutdown_connections 2024-04-25T10:59:49.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.508+0000 7f7018d5e640 1 --2- 172.21.15.69:0/2600765669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ffc005250 0x7f6ffc00b9c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:49.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.508+0000 7f7018d5e640 1 -- 172.21.15.69:0/2600765669 >> 172.21.15.69:0/2600765669 conn(0x7f701408d110 msgr2=0x7f701408d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:49.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.508+0000 7f7018d5e640 1 -- 172.21.15.69:0/2600765669 shutdown_connections 2024-04-25T10:59:49.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.508+0000 7f7018d5e640 1 -- 172.21.15.69:0/2600765669 wait complete. 2024-04-25T10:59:49.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.510+0000 7f7018d5e640 1 Processor -- start 2024-04-25T10:59:49.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.510+0000 7f7018d5e640 1 -- start start 2024-04-25T10:59:49.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.511+0000 7f7018d5e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ffc005250 0x7f70141e8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:49.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.511+0000 7f7018d5e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f701415efe0 0x7f7014160950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:49.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.511+0000 7f7018d5e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7014160e90 0x7f701415ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:49.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.511+0000 7f7018d5e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f70140b38e0 con 0x7f7014160e90 2024-04-25T10:59:49.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.511+0000 7f7018d5e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f70140b4820 con 0x7f701415efe0 2024-04-25T10:59:49.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.511+0000 7f7018d5e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f70140d3a20 con 0x7f6ffc005250 2024-04-25T10:59:49.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.511+0000 7f7012d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7014160e90 0x7f701415ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:49.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.511+0000 7f7012d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7014160e90 0x7f701415ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55096/0 (socket says 172.21.15.69:55096) 2024-04-25T10:59:49.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.511+0000 7f7012d9d640 1 -- 172.21.15.69:0/1651635057 learned_addr learned my addr 172.21.15.69:0/1651635057 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.511+0000 7f701259c640 1 --2- 172.21.15.69:0/1651635057 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ffc005250 0x7f70141e8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.511+0000 7f7011d9b640 1 --2- 172.21.15.69:0/1651635057 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f701415efe0 0x7f7014160950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.511+0000 7f7012d9d640 1 -- 172.21.15.69:0/1651635057 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ffc005250 msgr2=0x7f70141e8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.511+0000 7f7012d9d640 1 --2- 172.21.15.69:0/1651635057 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ffc005250 0x7f70141e8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.511+0000 7f7012d9d640 1 -- 172.21.15.69:0/1651635057 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f701415efe0 msgr2=0x7f7014160950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.511+0000 7f7012d9d640 1 --2- 172.21.15.69:0/1651635057 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f701415efe0 0x7f7014160950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.511+0000 7f7012d9d640 1 -- 172.21.15.69:0/1651635057 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70141612a0 con 0x7f7014160e90 2024-04-25T10:59:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.511+0000 7f701259c640 1 --2- 172.21.15.69:0/1651635057 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ffc005250 0x7f70141e8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.511+0000 7f7011d9b640 1 --2- 172.21.15.69:0/1651635057 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f701415efe0 0x7f7014160950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.512+0000 7f7012d9d640 1 --2- 172.21.15.69:0/1651635057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7014160e90 0x7f701415ce30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f700c005040 tx=0x7f700c001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.512+0000 7f7002ffd640 1 -- 172.21.15.69:0/1651635057 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c007a30 con 0x7f7014160e90 2024-04-25T10:59:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.512+0000 7f7002ffd640 1 -- 172.21.15.69:0/1651635057 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f700c000ed0 con 0x7f7014160e90 2024-04-25T10:59:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.512+0000 7f7018d5e640 1 -- 172.21.15.69:0/1651635057 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70141e74e0 con 0x7f7014160e90 2024-04-25T10:59:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.512+0000 7f7002ffd640 1 -- 172.21.15.69:0/1651635057 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c007230 con 0x7f7014160e90 2024-04-25T10:59:49.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.512+0000 7f7018d5e640 1 -- 172.21.15.69:0/1651635057 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70141e7aa0 con 0x7f7014160e90 2024-04-25T10:59:49.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.513+0000 7f7018d5e640 1 -- 172.21.15.69:0/1651635057 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7014050740 con 0x7f7014160e90 2024-04-25T10:59:49.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.513+0000 7f7002ffd640 1 -- 172.21.15.69:0/1651635057 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f700c0052d0 con 0x7f7014160e90 2024-04-25T10:59:49.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.514+0000 7f7002ffd640 1 --2- 172.21.15.69:0/1651635057 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f6ff40744e0 0x7f6ff40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:49.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.514+0000 7f7002ffd640 1 -- 172.21.15.69:0/1651635057 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f700c0b48c0 con 0x7f7014160e90 2024-04-25T10:59:49.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.514+0000 7f701259c640 1 --2- 172.21.15.69:0/1651635057 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f6ff40744e0 0x7f6ff40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:49.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.514+0000 7f701259c640 1 --2- 172.21.15.69:0/1651635057 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f6ff40744e0 0x7f6ff40769f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f70140744c0 tx=0x7f7004006040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:49.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.518+0000 7f7002ffd640 1 -- 172.21.15.69:0/1651635057 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f70141612a0 con 0x7f7014160e90 2024-04-25T10:59:49.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.704+0000 7f7018d5e640 1 -- 172.21.15.69:0/1651635057 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f70140b52c0 con 0x7f7014160e90 2024-04-25T10:59:49.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.704+0000 7f7002ffd640 1 -- 172.21.15.69:0/1651635057 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f700c02ae20 con 0x7f7014160e90 2024-04-25T10:59:49.707 INFO:teuthology.orchestra.run.smithi069.stderr:listed 0 entries 2024-04-25T10:59:49.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.706+0000 7f7018d5e640 1 -- 172.21.15.69:0/1651635057 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f6ff40744e0 msgr2=0x7f6ff40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:49.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.706+0000 7f7018d5e640 1 --2- 172.21.15.69:0/1651635057 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f6ff40744e0 0x7f6ff40769f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f70140744c0 tx=0x7f7004006040 comp rx=0 tx=0).stop 2024-04-25T10:59:49.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.706+0000 7f7018d5e640 1 -- 172.21.15.69:0/1651635057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7014160e90 msgr2=0x7f701415ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:49.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.706+0000 7f7018d5e640 1 --2- 172.21.15.69:0/1651635057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7014160e90 0x7f701415ce30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f700c005040 tx=0x7f700c001190 comp rx=0 tx=0).stop 2024-04-25T10:59:49.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.706+0000 7f7018d5e640 1 -- 172.21.15.69:0/1651635057 shutdown_connections 2024-04-25T10:59:49.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.706+0000 7f7018d5e640 1 --2- 172.21.15.69:0/1651635057 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f6ff40744e0 0x7f6ff40769f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:49.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.706+0000 7f7018d5e640 1 --2- 172.21.15.69:0/1651635057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7014160e90 0x7f701415ce30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:49.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.706+0000 7f7018d5e640 1 --2- 172.21.15.69:0/1651635057 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f701415efe0 0x7f7014160950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:49.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.706+0000 7f7018d5e640 1 --2- 172.21.15.69:0/1651635057 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ffc005250 0x7f70141e8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:49.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.706+0000 7f7018d5e640 1 -- 172.21.15.69:0/1651635057 >> 172.21.15.69:0/1651635057 conn(0x7f701408d110 msgr2=0x7f70140684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:49.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.706+0000 7f7018d5e640 1 -- 172.21.15.69:0/1651635057 shutdown_connections 2024-04-25T10:59:49.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:49.706+0000 7f7018d5e640 1 -- 172.21.15.69:0/1651635057 wait complete. 2024-04-25T10:59:49.718 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T10:59:49.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:49.718 DEBUG:teuthology.orchestra.run.smithi146:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-25T10:59:49.747 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2024-04-25T10:59:49.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:49.748 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:59:49.815 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:59:49.815 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:59:49.830 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:59:49.830 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:59:49.885 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T10:59:49.886 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:59:49.941 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:59:50.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.087+0000 7fd1cf89f640 1 Processor -- start 2024-04-25T10:59:50.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.087+0000 7fd1cf89f640 1 -- start start 2024-04-25T10:59:50.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.089+0000 7fd1cf89f640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd1c80b4830 0x7fd1c806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:50.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.089+0000 7fd1cf89f640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fd1c80b2dc0 0x7fd1c80b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:50.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.089+0000 7fd1cf89f640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1c806d870 con 0x7fd1c80d4570 2024-04-25T10:59:50.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.089+0000 7fd1cf89f640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fd1c80b4c40 con 0x7fd1c80b2dc0 2024-04-25T10:59:50.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.089+0000 7fd1cf89f640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fd1c806e460 con 0x7fd1c80b4830 2024-04-25T10:59:50.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.089+0000 7fd1cde3c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fd1c80b2dc0 0x7fd1c80b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:50.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.089+0000 7fd1cd63b640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fd1c80d4570 0x7fd1c80d04b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:50164/0 (socket says 172.21.15.69:50164) 2024-04-25T10:59:50.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.089+0000 7fd1cd63b640 1 -- 172.21.15.69:0/2580504964 learned_addr learned my addr 172.21.15.69:0/2580504964 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:50.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.089+0000 7fd1cde3c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fd1c80b2dc0 0x7fd1c80b31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55112/0 (socket says 172.21.15.69:55112) 2024-04-25T10:59:50.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.089+0000 7fd1cce3a640 1 --2- 172.21.15.69:0/2580504964 >> v2:172.21.15.146:3300/0 conn(0x7fd1c80b4830 0x7fd1c806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:50.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.089+0000 7fd1cde3c640 1 -- 172.21.15.69:0/2580504964 >> v2:172.21.15.146:3300/0 conn(0x7fd1c80b4830 msgr2=0x7fd1c806e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:50.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.089+0000 7fd1cde3c640 1 --2- 172.21.15.69:0/2580504964 >> v2:172.21.15.146:3300/0 conn(0x7fd1c80b4830 0x7fd1c806e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:50.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.089+0000 7fd1cde3c640 1 -- 172.21.15.69:0/2580504964 >> v1:172.21.15.69:6789/0 conn(0x7fd1c80d4570 legacy=0x7fd1c80d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:50.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.089+0000 7fd1bffff640 1 -- 172.21.15.69:0/2580504964 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3418299220 0 0) 0x7fd1c806d870 con 0x7fd1c80d4570 2024-04-25T10:59:50.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.089+0000 7fd1cde3c640 1 -- 172.21.15.69:0/2580504964 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c806db10 con 0x7fd1c80b2dc0 2024-04-25T10:59:50.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.090+0000 7fd1cde3c640 1 --2- 172.21.15.69:0/2580504964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1c80b2dc0 0x7fd1c80b31d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd1c806e5e0 tx=0x7fd1c002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab73b8bdc712db5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:50.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.090+0000 7fd1bffff640 1 -- 172.21.15.69:0/2580504964 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0037070 con 0x7fd1c80b2dc0 2024-04-25T10:59:50.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.090+0000 7fd1bffff640 1 -- 172.21.15.69:0/2580504964 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1c0032440 con 0x7fd1c80b2dc0 2024-04-25T10:59:50.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.090+0000 7fd1bffff640 1 -- 172.21.15.69:0/2580504964 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c002ae60 con 0x7fd1c80b2dc0 2024-04-25T10:59:50.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.091+0000 7fd1cf89f640 1 -- 172.21.15.69:0/2580504964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1c80b2dc0 msgr2=0x7fd1c80b31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:50.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.091+0000 7fd1cf89f640 1 --2- 172.21.15.69:0/2580504964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1c80b2dc0 0x7fd1c80b31d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd1c806e5e0 tx=0x7fd1c002a110 comp rx=0 tx=0).stop 2024-04-25T10:59:50.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.091+0000 7fd1cf89f640 1 -- 172.21.15.69:0/2580504964 shutdown_connections 2024-04-25T10:59:50.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.091+0000 7fd1cf89f640 1 --2- 172.21.15.69:0/2580504964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1c80b2dc0 0x7fd1c80b31d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:50.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.091+0000 7fd1cf89f640 1 --2- 172.21.15.69:0/2580504964 >> v2:172.21.15.146:3300/0 conn(0x7fd1c80b4830 0x7fd1c806e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:50.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.091+0000 7fd1cf89f640 1 -- 172.21.15.69:0/2580504964 >> 172.21.15.69:0/2580504964 conn(0x7fd1c80d3a50 msgr2=0x7fd1c80d3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:50.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.091+0000 7fd1cf89f640 1 -- 172.21.15.69:0/2580504964 shutdown_connections 2024-04-25T10:59:50.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.091+0000 7fd1cf89f640 1 -- 172.21.15.69:0/2580504964 wait complete. 2024-04-25T10:59:50.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.093+0000 7fd1cf89f640 1 Processor -- start 2024-04-25T10:59:50.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.093+0000 7fd1cf89f640 1 -- start start 2024-04-25T10:59:50.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.094+0000 7fd1cf89f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1c80b2dc0 0x7fd1c80c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:50.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.094+0000 7fd1cf89f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1c80b4830 0x7fd1c80b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:50.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.094+0000 7fd1cf89f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1c80d4570 0x7fd1c80b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:50.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.094+0000 7fd1cf89f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c80d0fb0 con 0x7fd1c80b4830 2024-04-25T10:59:50.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.094+0000 7fd1cf89f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c80d0bc0 con 0x7fd1c80b2dc0 2024-04-25T10:59:50.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.094+0000 7fd1cf89f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd1c80b3710 con 0x7fd1c80d4570 2024-04-25T10:59:50.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.094+0000 7fd1cce3a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1c80b4830 0x7fd1c80b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:50.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.094+0000 7fd1cce3a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1c80b4830 0x7fd1c80b1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55116/0 (socket says 172.21.15.69:55116) 2024-04-25T10:59:50.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.094+0000 7fd1cce3a640 1 -- 172.21.15.69:0/2577769538 learned_addr learned my addr 172.21.15.69:0/2577769538 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:50.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.094+0000 7fd1cd63b640 1 --2- 172.21.15.69:0/2577769538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1c80b2dc0 0x7fd1c80c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:50.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.094+0000 7fd1cce3a640 1 -- 172.21.15.69:0/2577769538 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1c80d4570 msgr2=0x7fd1c80b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:50.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.094+0000 7fd1cde3c640 1 --2- 172.21.15.69:0/2577769538 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1c80d4570 0x7fd1c80b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:50.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.094+0000 7fd1cce3a640 1 --2- 172.21.15.69:0/2577769538 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1c80d4570 0x7fd1c80b1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:50.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.094+0000 7fd1cce3a640 1 -- 172.21.15.69:0/2577769538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1c80b2dc0 msgr2=0x7fd1c80c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:50.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.094+0000 7fd1cce3a640 1 --2- 172.21.15.69:0/2577769538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1c80b2dc0 0x7fd1c80c7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:50.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.094+0000 7fd1cce3a640 1 -- 172.21.15.69:0/2577769538 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c81f95d0 con 0x7fd1c80b4830 2024-04-25T10:59:50.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.094+0000 7fd1cd63b640 1 --2- 172.21.15.69:0/2577769538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1c80b2dc0 0x7fd1c80c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:50.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.094+0000 7fd1cde3c640 1 --2- 172.21.15.69:0/2577769538 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1c80d4570 0x7fd1c80b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:50.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.095+0000 7fd1cce3a640 1 --2- 172.21.15.69:0/2577769538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1c80b4830 0x7fd1c80b1690 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd1b800c5a0 tx=0x7fd1b800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:50.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.095+0000 7fd1bdffb640 1 -- 172.21.15.69:0/2577769538 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8015070 con 0x7fd1c80b4830 2024-04-25T10:59:50.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.095+0000 7fd1bdffb640 1 -- 172.21.15.69:0/2577769538 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1b8004220 con 0x7fd1c80b4830 2024-04-25T10:59:50.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.095+0000 7fd1cf89f640 1 -- 172.21.15.69:0/2577769538 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1c81f9890 con 0x7fd1c80b4830 2024-04-25T10:59:50.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.095+0000 7fd1bdffb640 1 -- 172.21.15.69:0/2577769538 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8003610 con 0x7fd1c80b4830 2024-04-25T10:59:50.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.095+0000 7fd1cf89f640 1 -- 172.21.15.69:0/2577769538 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1c81f9e50 con 0x7fd1c80b4830 2024-04-25T10:59:50.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.096+0000 7fd1cf89f640 1 -- 172.21.15.69:0/2577769538 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1c809e5d0 con 0x7fd1c80b4830 2024-04-25T10:59:50.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.096+0000 7fd1bdffb640 1 -- 172.21.15.69:0/2577769538 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd1b80037b0 con 0x7fd1c80b4830 2024-04-25T10:59:50.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.097+0000 7fd1bdffb640 1 --2- 172.21.15.69:0/2577769538 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd1ac0744b0 0x7fd1ac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:50.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.097+0000 7fd1bdffb640 1 -- 172.21.15.69:0/2577769538 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd1b8095260 con 0x7fd1c80b4830 2024-04-25T10:59:50.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.097+0000 7fd1cd63b640 1 --2- 172.21.15.69:0/2577769538 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd1ac0744b0 0x7fd1ac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:50.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.097+0000 7fd1cd63b640 1 --2- 172.21.15.69:0/2577769538 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd1ac0744b0 0x7fd1ac0769c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4001d50 tx=0x7fd1c4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:50.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.101+0000 7fd1bdffb640 1 -- 172.21.15.69:0/2577769538 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd1b805fa90 con 0x7fd1c80b4830 2024-04-25T10:59:50.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.275+0000 7fd1cf89f640 1 -- 172.21.15.69:0/2577769538 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd1c8050740 con 0x7fd1c80b4830 2024-04-25T10:59:50.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.276+0000 7fd1bdffb640 1 -- 172.21.15.69:0/2577769538 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7fd1b805f390 con 0x7fd1c80b4830 2024-04-25T10:59:50.277 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:50.277 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":20,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T10:59:20.630151+0000","last_up_change":"2024-04-25T10:59:05.546839+0000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:59:06.422674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:59:16.727980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:59:17.938486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6825","nonce":1876101065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6827","nonce":1876101065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6831","nonce":1876101065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6829","nonce":1876101065}]},"public_addr":"172.21.15.69:6825/1876101065","cluster_addr":"172.21.15.69:6827/1876101065","heartbeat_back_addr":"172.21.15.69:6831/1876101065","heartbeat_front_addr":"172.21.15.69:6829/1876101065","state":["exists","up"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6801","nonce":1806438563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6803","nonce":1806438563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6807","nonce":1806438563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6805","nonce":1806438563}]},"public_addr":"172.21.15.69:6801/1806438563","cluster_addr":"172.21.15.69:6803/1806438563","heartbeat_back_addr":"172.21.15.69:6807/1806438563","heartbeat_front_addr":"172.21.15.69:6805/1806438563","state":["exists","up"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6817","nonce":1593617983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6819","nonce":1593617983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6823","nonce":1593617983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6821","nonce":1593617983}]},"public_addr":"172.21.15.69:6817/1593617983","cluster_addr":"172.21.15.69:6819/1593617983","heartbeat_back_addr":"172.21.15.69:6823/1593617983","heartbeat_front_addr":"172.21.15.69:6821/1593617983","state":["exists","up"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6809","nonce":3950661294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6811","nonce":3950661294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6815","nonce":3950661294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6813","nonce":3950661294}]},"public_addr":"172.21.15.69:6809/3950661294","cluster_addr":"172.21.15.69:6811/3950661294","heartbeat_back_addr":"172.21.15.69:6815/3950661294","heartbeat_front_addr":"172.21.15.69:6813/3950661294","state":["exists","up"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6801","nonce":2508929872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6803","nonce":2508929872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6807","nonce":2508929872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6805","nonce":2508929872}]},"public_addr":"172.21.15.146:6801/2508929872","cluster_addr":"172.21.15.146:6803/2508929872","heartbeat_back_addr":"172.21.15.146:6807/2508929872","heartbeat_front_addr":"172.21.15.146:6805/2508929872","state":["exists","up"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6811","nonce":1310911138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6816","nonce":1310911138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6823","nonce":1310911138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6821","nonce":1310911138}]},"public_addr":"172.21.15.146:6811/1310911138","cluster_addr":"172.21.15.146:6816/1310911138","heartbeat_back_addr":"172.21.15.146:6823/1310911138","heartbeat_front_addr":"172.21.15.146:6821/1310911138","state":["exists","up"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6810","nonce":634688021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6813","nonce":634688021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6820","nonce":634688021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6817","nonce":634688021}]},"public_addr":"172.21.15.146:6810/634688021","cluster_addr":"172.21.15.146:6813/634688021","heartbeat_back_addr":"172.21.15.146:6820/634688021","heartbeat_front_addr":"172.21.15.146:6817/634688021","state":["exists","up"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6825","nonce":3814215935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6827","nonce":3814215935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6831","nonce":3814215935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6829","nonce":3814215935}]},"public_addr":"172.21.15.146:6825/3814215935","cluster_addr":"172.21.15.146:6827/3814215935","heartbeat_back_addr":"172.21.15.146:6831/3814215935","heartbeat_front_addr":"172.21.15.146:6829/3814215935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.820200+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.850189+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.764560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.866248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.807897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.869881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.782310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.827131+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:59:50.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.278+0000 7fd1cf89f640 1 -- 172.21.15.69:0/2577769538 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd1ac0744b0 msgr2=0x7fd1ac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:50.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.278+0000 7fd1cf89f640 1 --2- 172.21.15.69:0/2577769538 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd1ac0744b0 0x7fd1ac0769c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4001d50 tx=0x7fd1c4009040 comp rx=0 tx=0).stop 2024-04-25T10:59:50.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.278+0000 7fd1cf89f640 1 -- 172.21.15.69:0/2577769538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1c80b4830 msgr2=0x7fd1c80b1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:50.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.278+0000 7fd1cf89f640 1 --2- 172.21.15.69:0/2577769538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1c80b4830 0x7fd1c80b1690 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd1b800c5a0 tx=0x7fd1b800ca60 comp rx=0 tx=0).stop 2024-04-25T10:59:50.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.278+0000 7fd1cf89f640 1 -- 172.21.15.69:0/2577769538 shutdown_connections 2024-04-25T10:59:50.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.278+0000 7fd1cf89f640 1 --2- 172.21.15.69:0/2577769538 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd1ac0744b0 0x7fd1ac0769c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:50.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.278+0000 7fd1cf89f640 1 --2- 172.21.15.69:0/2577769538 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1c80d4570 0x7fd1c80b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:50.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.278+0000 7fd1cf89f640 1 --2- 172.21.15.69:0/2577769538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1c80b4830 0x7fd1c80b1690 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:50.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.278+0000 7fd1cf89f640 1 --2- 172.21.15.69:0/2577769538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1c80b2dc0 0x7fd1c80c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:50.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.278+0000 7fd1cf89f640 1 -- 172.21.15.69:0/2577769538 >> 172.21.15.69:0/2577769538 conn(0x7fd1c80d3a50 msgr2=0x7fd1c8164010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:50.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.278+0000 7fd1cf89f640 1 -- 172.21.15.69:0/2577769538 shutdown_connections 2024-04-25T10:59:50.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.278+0000 7fd1cf89f640 1 -- 172.21.15.69:0/2577769538 wait complete. 2024-04-25T10:59:50.290 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:59:06.422674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:59:16.727980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:59:17.938486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:59:50.290 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:59:50.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.399+0000 7f87d55c0640 1 Processor -- start 2024-04-25T10:59:50.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.399+0000 7f87d55c0640 1 -- start start 2024-04-25T10:59:50.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.401+0000 7f87d55c0640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f87d00d1e20 0x7f87d00d3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:50.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.401+0000 7f87d55c0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f87d0164020 0x7f87d0164430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:50.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.401+0000 7f87d55c0640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87d0164970 con 0x7f87d00b2e60 2024-04-25T10:59:50.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.401+0000 7f87d55c0640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f87d00d23f0 con 0x7f87d00d1e20 2024-04-25T10:59:50.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.401+0000 7f87d55c0640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f87d00d26d0 con 0x7f87d0164020 2024-04-25T10:59:50.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.401+0000 7f87ceffd640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f87d00d1e20 0x7f87d00d3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:50.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.401+0000 7f87ceffd640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f87d00d1e20 0x7f87d00d3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55132/0 (socket says 172.21.15.69:55132) 2024-04-25T10:59:50.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.401+0000 7f87ceffd640 1 -- 172.21.15.69:0/4173927999 learned_addr learned my addr 172.21.15.69:0/4173927999 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:50.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.401+0000 7f87ce7fc640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f87d00b2e60 0x7f87d008d770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:50174/0 (socket says 172.21.15.69:50174) 2024-04-25T10:59:50.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.401+0000 7f87cf7fe640 1 --2- 172.21.15.69:0/4173927999 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f87d0164020 0x7f87d0164430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:50.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.401+0000 7f87ceffd640 1 -- 172.21.15.69:0/4173927999 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f87d0164020 msgr2=0x7f87d0164430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:50.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.401+0000 7f87ceffd640 1 --2- 172.21.15.69:0/4173927999 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f87d0164020 0x7f87d0164430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:50.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.401+0000 7f87ceffd640 1 -- 172.21.15.69:0/4173927999 >> v1:172.21.15.69:6789/0 conn(0x7f87d00b2e60 legacy=0x7f87d008d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:50.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.401+0000 7f87ceffd640 1 -- 172.21.15.69:0/4173927999 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d00d29b0 con 0x7f87d00d1e20 2024-04-25T10:59:50.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.401+0000 7f87cdffb640 1 -- 172.21.15.69:0/4173927999 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1137217277 0 0) 0x7f87c4001110 con 0x7f87d00b2e60 2024-04-25T10:59:50.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.402+0000 7f87ceffd640 1 --2- 172.21.15.69:0/4173927999 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87d00d1e20 0x7f87d00d3490 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f87c0009e50 tx=0x7f87c002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80dfd6a94b8ed796 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:50.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.402+0000 7f87cdffb640 1 -- 172.21.15.69:0/4173927999 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c0036070 con 0x7f87d00d1e20 2024-04-25T10:59:50.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.402+0000 7f87cdffb640 1 -- 172.21.15.69:0/4173927999 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87c0031600 con 0x7f87d00d1e20 2024-04-25T10:59:50.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.402+0000 7f87cdffb640 1 -- 172.21.15.69:0/4173927999 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c00319d0 con 0x7f87d00d1e20 2024-04-25T10:59:50.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.403+0000 7f87d55c0640 1 -- 172.21.15.69:0/4173927999 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87d00d1e20 msgr2=0x7f87d00d3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:50.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.403+0000 7f87d55c0640 1 --2- 172.21.15.69:0/4173927999 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87d00d1e20 0x7f87d00d3490 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f87c0009e50 tx=0x7f87c002a7a0 comp rx=0 tx=0).stop 2024-04-25T10:59:50.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.403+0000 7f87d55c0640 1 -- 172.21.15.69:0/4173927999 shutdown_connections 2024-04-25T10:59:50.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.403+0000 7f87d55c0640 1 --2- 172.21.15.69:0/4173927999 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f87d0164020 0x7f87d0164430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:50.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.403+0000 7f87d55c0640 1 --2- 172.21.15.69:0/4173927999 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87d00d1e20 0x7f87d00d3490 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:50.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.403+0000 7f87d55c0640 1 -- 172.21.15.69:0/4173927999 >> 172.21.15.69:0/4173927999 conn(0x7f87d00d0530 msgr2=0x7f87d00d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:50.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.403+0000 7f87d55c0640 1 -- 172.21.15.69:0/4173927999 shutdown_connections 2024-04-25T10:59:50.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.403+0000 7f87d55c0640 1 -- 172.21.15.69:0/4173927999 wait complete. 2024-04-25T10:59:50.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.405+0000 7f87d55c0640 1 Processor -- start 2024-04-25T10:59:50.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.405+0000 7f87d55c0640 1 -- start start 2024-04-25T10:59:50.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.406+0000 7f87d55c0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87d00b2e60 0x7f87d015e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:50.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.406+0000 7f87d55c0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f87d00d1e20 0x7f87d01fa940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:50.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.406+0000 7f87d55c0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87d0164020 0x7f87d00cc940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:50.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.406+0000 7f87d55c0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f87d00ce070 con 0x7f87d00b2e60 2024-04-25T10:59:50.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.406+0000 7f87d55c0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f87d00d2f70 con 0x7f87d0164020 2024-04-25T10:59:50.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.406+0000 7f87d55c0640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f87d00ce1f0 con 0x7f87d00d1e20 2024-04-25T10:59:50.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.406+0000 7f87ceffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87d00b2e60 0x7f87d015e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:50.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.406+0000 7f87ceffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87d00b2e60 0x7f87d015e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55140/0 (socket says 172.21.15.69:55140) 2024-04-25T10:59:50.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.406+0000 7f87ceffd640 1 -- 172.21.15.69:0/2607105430 learned_addr learned my addr 172.21.15.69:0/2607105430 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:50.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.406+0000 7f87ce7fc640 1 --2- 172.21.15.69:0/2607105430 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f87d00d1e20 0x7f87d01fa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:50.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.406+0000 7f87cf7fe640 1 --2- 172.21.15.69:0/2607105430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87d0164020 0x7f87d00cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:50.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.406+0000 7f87ceffd640 1 -- 172.21.15.69:0/2607105430 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f87d00d1e20 msgr2=0x7f87d01fa940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:50.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.406+0000 7f87ceffd640 1 --2- 172.21.15.69:0/2607105430 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f87d00d1e20 0x7f87d01fa940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:50.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.407+0000 7f87ceffd640 1 -- 172.21.15.69:0/2607105430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87d0164020 msgr2=0x7f87d00cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:50.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.407+0000 7f87ceffd640 1 --2- 172.21.15.69:0/2607105430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87d0164020 0x7f87d00cc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:50.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.407+0000 7f87ceffd640 1 -- 172.21.15.69:0/2607105430 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d00c76f0 con 0x7f87d00b2e60 2024-04-25T10:59:50.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.407+0000 7f87ceffd640 1 --2- 172.21.15.69:0/2607105430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87d00b2e60 0x7f87d015e550 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f87c0009e50 tx=0x7f87c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:50.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.407+0000 7f87b77fe640 1 -- 172.21.15.69:0/2607105430 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c0036050 con 0x7f87d00b2e60 2024-04-25T10:59:50.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.407+0000 7f87b77fe640 1 -- 172.21.15.69:0/2607105430 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87c0042070 con 0x7f87d00b2e60 2024-04-25T10:59:50.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.407+0000 7f87b77fe640 1 -- 172.21.15.69:0/2607105430 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c0009510 con 0x7f87d00b2e60 2024-04-25T10:59:50.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.407+0000 7f87d55c0640 1 -- 172.21.15.69:0/2607105430 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87d01fb430 con 0x7f87d00b2e60 2024-04-25T10:59:50.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.407+0000 7f87d55c0640 1 -- 172.21.15.69:0/2607105430 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87d00c7990 con 0x7f87d00b2e60 2024-04-25T10:59:50.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.408+0000 7f87d55c0640 1 -- 172.21.15.69:0/2607105430 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87d0050740 con 0x7f87d00b2e60 2024-04-25T10:59:50.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.409+0000 7f87b77fe640 1 -- 172.21.15.69:0/2607105430 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f87c0007c50 con 0x7f87d00b2e60 2024-04-25T10:59:50.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.409+0000 7f87b77fe640 1 --2- 172.21.15.69:0/2607105430 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f87bc0744b0 0x7f87bc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:50.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.409+0000 7f87b77fe640 1 -- 172.21.15.69:0/2607105430 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f87c00b5ff0 con 0x7f87d00b2e60 2024-04-25T10:59:50.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.409+0000 7f87ce7fc640 1 --2- 172.21.15.69:0/2607105430 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f87bc0744b0 0x7f87bc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:50.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.410+0000 7f87ce7fc640 1 --2- 172.21.15.69:0/2607105430 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f87bc0744b0 0x7f87bc0769c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f87c4007fd0 tx=0x7f87c4007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:50.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.414+0000 7f87b77fe640 1 -- 172.21.15.69:0/2607105430 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f87c0080130 con 0x7f87d00b2e60 2024-04-25T10:59:50.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.588+0000 7f87d55c0640 1 -- 172.21.15.69:0/2607105430 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f87d0072f90 con 0x7f87d00b2e60 2024-04-25T10:59:50.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.589+0000 7f87b77fe640 1 -- 172.21.15.69:0/2607105430 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f87c0084430 con 0x7f87d00b2e60 2024-04-25T10:59:50.589 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-25T10:59:50.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.590+0000 7f87d55c0640 1 -- 172.21.15.69:0/2607105430 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f87bc0744b0 msgr2=0x7f87bc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:50.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.590+0000 7f87d55c0640 1 --2- 172.21.15.69:0/2607105430 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f87bc0744b0 0x7f87bc0769c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f87c4007fd0 tx=0x7f87c4007de0 comp rx=0 tx=0).stop 2024-04-25T10:59:50.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.590+0000 7f87d55c0640 1 -- 172.21.15.69:0/2607105430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87d00b2e60 msgr2=0x7f87d015e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:50.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.590+0000 7f87d55c0640 1 --2- 172.21.15.69:0/2607105430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87d00b2e60 0x7f87d015e550 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f87c0009e50 tx=0x7f87c0004a40 comp rx=0 tx=0).stop 2024-04-25T10:59:50.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.590+0000 7f87d55c0640 1 -- 172.21.15.69:0/2607105430 shutdown_connections 2024-04-25T10:59:50.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.591+0000 7f87d55c0640 1 --2- 172.21.15.69:0/2607105430 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f87bc0744b0 0x7f87bc0769c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:50.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.591+0000 7f87d55c0640 1 --2- 172.21.15.69:0/2607105430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87d0164020 0x7f87d00cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:50.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.591+0000 7f87d55c0640 1 --2- 172.21.15.69:0/2607105430 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f87d00d1e20 0x7f87d01fa940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:50.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.591+0000 7f87d55c0640 1 --2- 172.21.15.69:0/2607105430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87d00b2e60 0x7f87d015e550 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:50.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.591+0000 7f87d55c0640 1 -- 172.21.15.69:0/2607105430 >> 172.21.15.69:0/2607105430 conn(0x7f87d00d0530 msgr2=0x7f87d00cf7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:50.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.591+0000 7f87d55c0640 1 -- 172.21.15.69:0/2607105430 shutdown_connections 2024-04-25T10:59:50.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.591+0000 7f87d55c0640 1 -- 172.21.15.69:0/2607105430 wait complete. 2024-04-25T10:59:50.603 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:59:50.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.714+0000 7f3c957e3640 1 Processor -- start 2024-04-25T10:59:50.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.714+0000 7f3c957e3640 1 -- start start 2024-04-25T10:59:50.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.715+0000 7f3c957e3640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3c900d0e00 0x7f3c900d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:50.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.715+0000 7f3c957e3640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f3c9006c5c0 0x7f3c9006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:50.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.715+0000 7f3c957e3640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c9006cf10 con 0x7f3c900b7810 2024-04-25T10:59:50.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.715+0000 7f3c957e3640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f3c900d5a60 con 0x7f3c9006c5c0 2024-04-25T10:59:50.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.715+0000 7f3c957e3640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f3c900d1850 con 0x7f3c900d0e00 2024-04-25T10:59:50.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.715+0000 7f3c87fff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f3c9006c5c0 0x7f3c9006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:50.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.715+0000 7f3c87fff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f3c9006c5c0 0x7f3c9006c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55148/0 (socket says 172.21.15.69:55148) 2024-04-25T10:59:50.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.715+0000 7f3c87fff640 1 -- 172.21.15.69:0/2201903812 learned_addr learned my addr 172.21.15.69:0/2201903812 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:50.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.716+0000 7f3c8effd640 1 --2- 172.21.15.69:0/2201903812 >> v2:172.21.15.146:3300/0 conn(0x7f3c900d0e00 0x7f3c900d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:50.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.716+0000 7f3c87fff640 1 -- 172.21.15.69:0/2201903812 >> v2:172.21.15.146:3300/0 conn(0x7f3c900d0e00 msgr2=0x7f3c900d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:50.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.716+0000 7f3c87fff640 1 --2- 172.21.15.69:0/2201903812 >> v2:172.21.15.146:3300/0 conn(0x7f3c900d0e00 0x7f3c900d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:50.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.716+0000 7f3c87fff640 1 -- 172.21.15.69:0/2201903812 >> v1:172.21.15.146:6789/0 conn(0x7f3c900b7810 legacy=0x7f3c900b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:50.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.716+0000 7f3c87fff640 1 -- 172.21.15.69:0/2201903812 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c90160de0 con 0x7f3c9006c5c0 2024-04-25T10:59:50.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.716+0000 7f3c8effd640 1 --2- 172.21.15.69:0/2201903812 >> v2:172.21.15.146:3300/0 conn(0x7f3c900d0e00 0x7f3c900d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:50.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.716+0000 7f3c87fff640 1 --2- 172.21.15.69:0/2201903812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c9006c5c0 0x7f3c9006c9d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3c900d5be0 tx=0x7f3c7802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb50115d1e0aaadb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:50.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.717+0000 7f3c8e7fc640 1 -- 172.21.15.69:0/2201903812 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c78037070 con 0x7f3c9006c5c0 2024-04-25T10:59:50.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.717+0000 7f3c8e7fc640 1 -- 172.21.15.69:0/2201903812 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c78032440 con 0x7f3c9006c5c0 2024-04-25T10:59:50.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.717+0000 7f3c8e7fc640 1 -- 172.21.15.69:0/2201903812 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c78032810 con 0x7f3c9006c5c0 2024-04-25T10:59:50.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.717+0000 7f3c957e3640 1 -- 172.21.15.69:0/2201903812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c9006c5c0 msgr2=0x7f3c9006c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:50.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.717+0000 7f3c957e3640 1 --2- 172.21.15.69:0/2201903812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c9006c5c0 0x7f3c9006c9d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3c900d5be0 tx=0x7f3c7802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:59:50.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.717+0000 7f3c957e3640 1 -- 172.21.15.69:0/2201903812 shutdown_connections 2024-04-25T10:59:50.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.717+0000 7f3c957e3640 1 --2- 172.21.15.69:0/2201903812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c9006c5c0 0x7f3c9006c9d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:50.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.717+0000 7f3c957e3640 1 --2- 172.21.15.69:0/2201903812 >> v2:172.21.15.146:3300/0 conn(0x7f3c900d0e00 0x7f3c900d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:50.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.717+0000 7f3c957e3640 1 -- 172.21.15.69:0/2201903812 >> 172.21.15.69:0/2201903812 conn(0x7f3c900d5d90 msgr2=0x7f3c900d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:50.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.717+0000 7f3c957e3640 1 -- 172.21.15.69:0/2201903812 shutdown_connections 2024-04-25T10:59:50.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.717+0000 7f3c957e3640 1 -- 172.21.15.69:0/2201903812 wait complete. 2024-04-25T10:59:50.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.719+0000 7f3c957e3640 1 Processor -- start 2024-04-25T10:59:50.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.719+0000 7f3c957e3640 1 -- start start 2024-04-25T10:59:50.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.720+0000 7f3c957e3640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c9006c5c0 0x7f3c900ce920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:50.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.720+0000 7f3c957e3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c900b7810 0x7f3c9014c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:50.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.720+0000 7f3c957e3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c900d0e00 0x7f3c900cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:50.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.720+0000 7f3c957e3640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3c90160ad0 con 0x7f3c900b7810 2024-04-25T10:59:50.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.720+0000 7f3c957e3640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3c901613a0 con 0x7f3c900d0e00 2024-04-25T10:59:50.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.720+0000 7f3c957e3640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3c900b5320 con 0x7f3c9006c5c0 2024-04-25T10:59:50.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.720+0000 7f3c87fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c900b7810 0x7f3c9014c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:50.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.721+0000 7f3c87fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c900b7810 0x7f3c9014c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55164/0 (socket says 172.21.15.69:55164) 2024-04-25T10:59:50.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.721+0000 7f3c87fff640 1 -- 172.21.15.69:0/1934947088 learned_addr learned my addr 172.21.15.69:0/1934947088 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:50.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.721+0000 7f3c8effd640 1 --2- 172.21.15.69:0/1934947088 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c9006c5c0 0x7f3c900ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:50.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.721+0000 7f3c8f7fe640 1 --2- 172.21.15.69:0/1934947088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c900d0e00 0x7f3c900cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:50.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.721+0000 7f3c87fff640 1 -- 172.21.15.69:0/1934947088 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c9006c5c0 msgr2=0x7f3c900ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:50.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.721+0000 7f3c87fff640 1 --2- 172.21.15.69:0/1934947088 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c9006c5c0 0x7f3c900ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:50.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.721+0000 7f3c87fff640 1 -- 172.21.15.69:0/1934947088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c900d0e00 msgr2=0x7f3c900cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:50.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.721+0000 7f3c87fff640 1 --2- 172.21.15.69:0/1934947088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c900d0e00 0x7f3c900cb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:50.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.721+0000 7f3c87fff640 1 -- 172.21.15.69:0/1934947088 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c900cd880 con 0x7f3c900b7810 2024-04-25T10:59:50.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.721+0000 7f3c8f7fe640 1 --2- 172.21.15.69:0/1934947088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c900d0e00 0x7f3c900cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:50.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.721+0000 7f3c87fff640 1 --2- 172.21.15.69:0/1934947088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c900b7810 0x7f3c9014c200 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3c7802a220 tx=0x7f3c780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:50.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.721+0000 7f3c877fe640 1 -- 172.21.15.69:0/1934947088 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c78041020 con 0x7f3c900b7810 2024-04-25T10:59:50.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.721+0000 7f3c877fe640 1 -- 172.21.15.69:0/1934947088 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c780306c0 con 0x7f3c900b7810 2024-04-25T10:59:50.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.721+0000 7f3c957e3640 1 -- 172.21.15.69:0/1934947088 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c900cdb60 con 0x7f3c900b7810 2024-04-25T10:59:50.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.721+0000 7f3c877fe640 1 -- 172.21.15.69:0/1934947088 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c78037040 con 0x7f3c900b7810 2024-04-25T10:59:50.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.721+0000 7f3c957e3640 1 -- 172.21.15.69:0/1934947088 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c900ce120 con 0x7f3c900b7810 2024-04-25T10:59:50.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.722+0000 7f3c957e3640 1 -- 172.21.15.69:0/1934947088 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c90050740 con 0x7f3c900b7810 2024-04-25T10:59:50.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.723+0000 7f3c877fe640 1 -- 172.21.15.69:0/1934947088 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3c78044020 con 0x7f3c900b7810 2024-04-25T10:59:50.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.723+0000 7f3c877fe640 1 --2- 172.21.15.69:0/1934947088 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3c700744e0 0x7f3c700769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:50.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.724+0000 7f3c8effd640 1 --2- 172.21.15.69:0/1934947088 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3c700744e0 0x7f3c700769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:50.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.724+0000 7f3c877fe640 1 -- 172.21.15.69:0/1934947088 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3c780b5a60 con 0x7f3c900b7810 2024-04-25T10:59:50.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.724+0000 7f3c8effd640 1 --2- 172.21.15.69:0/1934947088 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3c700744e0 0x7f3c700769f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3c800096d0 tx=0x7f3c80008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:50.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.728+0000 7f3c877fe640 1 -- 172.21.15.69:0/1934947088 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3c7804b030 con 0x7f3c900b7810 2024-04-25T10:59:50.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.903+0000 7f3c957e3640 1 -- 172.21.15.69:0/1934947088 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3c900d4320 con 0x7f3c900b7810 2024-04-25T10:59:50.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.903+0000 7f3c877fe640 1 -- 172.21.15.69:0/1934947088 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f3c780801f0 con 0x7f3c900b7810 2024-04-25T10:59:50.904 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-25T10:59:50.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.905+0000 7f3c957e3640 1 -- 172.21.15.69:0/1934947088 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3c700744e0 msgr2=0x7f3c700769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:50.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.905+0000 7f3c957e3640 1 --2- 172.21.15.69:0/1934947088 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3c700744e0 0x7f3c700769f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3c800096d0 tx=0x7f3c80008040 comp rx=0 tx=0).stop 2024-04-25T10:59:50.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.905+0000 7f3c957e3640 1 -- 172.21.15.69:0/1934947088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c900b7810 msgr2=0x7f3c9014c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:50.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.905+0000 7f3c957e3640 1 --2- 172.21.15.69:0/1934947088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c900b7810 0x7f3c9014c200 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3c7802a220 tx=0x7f3c780046c0 comp rx=0 tx=0).stop 2024-04-25T10:59:50.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.905+0000 7f3c957e3640 1 -- 172.21.15.69:0/1934947088 shutdown_connections 2024-04-25T10:59:50.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.905+0000 7f3c957e3640 1 --2- 172.21.15.69:0/1934947088 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3c700744e0 0x7f3c700769f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:50.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.905+0000 7f3c957e3640 1 --2- 172.21.15.69:0/1934947088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c900d0e00 0x7f3c900cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:50.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.905+0000 7f3c957e3640 1 --2- 172.21.15.69:0/1934947088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c900b7810 0x7f3c9014c200 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:50.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.905+0000 7f3c957e3640 1 --2- 172.21.15.69:0/1934947088 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c9006c5c0 0x7f3c900ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:50.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.905+0000 7f3c957e3640 1 -- 172.21.15.69:0/1934947088 >> 172.21.15.69:0/1934947088 conn(0x7f3c900d5d90 msgr2=0x7f3c900d27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:50.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.906+0000 7f3c957e3640 1 -- 172.21.15.69:0/1934947088 shutdown_connections 2024-04-25T10:59:50.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:50.906+0000 7f3c957e3640 1 -- 172.21.15.69:0/1934947088 wait complete. 2024-04-25T10:59:50.917 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:59:51.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.027+0000 7f086faf9640 1 Processor -- start 2024-04-25T10:59:51.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.027+0000 7f086faf9640 1 -- start start 2024-04-25T10:59:51.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.028+0000 7f086faf9640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f08680d0190 0x7f08680d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:51.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.028+0000 7f086faf9640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f08680b7ef0 0x7f08680b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:51.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.028+0000 7f086faf9640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08680b3f30 con 0x7f08680d1f90 2024-04-25T10:59:51.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.028+0000 7f086faf9640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f08680b23e0 con 0x7f08680d0190 2024-04-25T10:59:51.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.028+0000 7f086faf9640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f08680b4210 con 0x7f08680b7ef0 2024-04-25T10:59:51.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.029+0000 7f086d895640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f08680d0190 0x7f08680d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:51.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.029+0000 7f086d895640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f08680d0190 0x7f08680d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55170/0 (socket says 172.21.15.69:55170) 2024-04-25T10:59:51.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.029+0000 7f086d895640 1 -- 172.21.15.69:0/2952197266 learned_addr learned my addr 172.21.15.69:0/2952197266 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:51.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.029+0000 7f086e096640 1 --2- 172.21.15.69:0/2952197266 >> v2:172.21.15.146:3300/0 conn(0x7f08680b7ef0 0x7f08680b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:51.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.029+0000 7f086d895640 1 -- 172.21.15.69:0/2952197266 >> v2:172.21.15.146:3300/0 conn(0x7f08680b7ef0 msgr2=0x7f08680b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:51.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.029+0000 7f086d895640 1 --2- 172.21.15.69:0/2952197266 >> v2:172.21.15.146:3300/0 conn(0x7f08680b7ef0 0x7f08680b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:51.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.029+0000 7f086d895640 1 -- 172.21.15.69:0/2952197266 >> v1:172.21.15.146:6789/0 conn(0x7f08680d1f90 legacy=0x7f08680b77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:51.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.029+0000 7f086d895640 1 -- 172.21.15.69:0/2952197266 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08680b2630 con 0x7f08680d0190 2024-04-25T10:59:51.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.030+0000 7f086d895640 1 --2- 172.21.15.69:0/2952197266 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08680d0190 0x7f08680d1a50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f086809c510 tx=0x7f086402a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75725489f450ce9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:51.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.030+0000 7f086c893640 1 -- 172.21.15.69:0/2952197266 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0864037070 con 0x7f08680d0190 2024-04-25T10:59:51.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.030+0000 7f086c893640 1 -- 172.21.15.69:0/2952197266 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0864032440 con 0x7f08680d0190 2024-04-25T10:59:51.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.030+0000 7f086c893640 1 -- 172.21.15.69:0/2952197266 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0864032810 con 0x7f08680d0190 2024-04-25T10:59:51.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.030+0000 7f086faf9640 1 -- 172.21.15.69:0/2952197266 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08680d0190 msgr2=0x7f08680d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:51.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.030+0000 7f086faf9640 1 --2- 172.21.15.69:0/2952197266 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08680d0190 0x7f08680d1a50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f086809c510 tx=0x7f086402a150 comp rx=0 tx=0).stop 2024-04-25T10:59:51.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.030+0000 7f086faf9640 1 -- 172.21.15.69:0/2952197266 shutdown_connections 2024-04-25T10:59:51.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.030+0000 7f086faf9640 1 --2- 172.21.15.69:0/2952197266 >> v2:172.21.15.146:3300/0 conn(0x7f08680b7ef0 0x7f08680b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:51.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.030+0000 7f086faf9640 1 --2- 172.21.15.69:0/2952197266 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08680d0190 0x7f08680d1a50 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:51.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.030+0000 7f086faf9640 1 -- 172.21.15.69:0/2952197266 >> 172.21.15.69:0/2952197266 conn(0x7f086809b7d0 msgr2=0x7f086809bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:51.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.031+0000 7f086faf9640 1 -- 172.21.15.69:0/2952197266 shutdown_connections 2024-04-25T10:59:51.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.031+0000 7f086faf9640 1 -- 172.21.15.69:0/2952197266 wait complete. 2024-04-25T10:59:51.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.032+0000 7f086faf9640 1 Processor -- start 2024-04-25T10:59:51.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.032+0000 7f086faf9640 1 -- start start 2024-04-25T10:59:51.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.033+0000 7f086faf9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08680d0190 0x7f08680b01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:51.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.034+0000 7f086faf9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08680d1f90 0x7f08680c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:51.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.034+0000 7f086faf9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08680b0730 0x7f086815ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:51.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.034+0000 7f086faf9640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f086806ee50 con 0x7f08680b0730 2024-04-25T10:59:51.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.034+0000 7f086faf9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f086806ecd0 con 0x7f08680d1f90 2024-04-25T10:59:51.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.034+0000 7f086faf9640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f086806efd0 con 0x7f08680d0190 2024-04-25T10:59:51.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.034+0000 7f086e096640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08680b0730 0x7f086815ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:51.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.034+0000 7f086e096640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08680b0730 0x7f086815ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55180/0 (socket says 172.21.15.69:55180) 2024-04-25T10:59:51.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.034+0000 7f086e096640 1 -- 172.21.15.69:0/2774468017 learned_addr learned my addr 172.21.15.69:0/2774468017 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:51.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.034+0000 7f086d895640 1 --2- 172.21.15.69:0/2774468017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08680d0190 0x7f08680b01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:51.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.034+0000 7f086d094640 1 --2- 172.21.15.69:0/2774468017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08680d1f90 0x7f08680c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:51.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.034+0000 7f086e096640 1 -- 172.21.15.69:0/2774468017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08680d0190 msgr2=0x7f08680b01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:51.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.034+0000 7f086e096640 1 --2- 172.21.15.69:0/2774468017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08680d0190 0x7f08680b01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:51.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.034+0000 7f086e096640 1 -- 172.21.15.69:0/2774468017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08680d1f90 msgr2=0x7f08680c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:51.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.034+0000 7f086e096640 1 --2- 172.21.15.69:0/2774468017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08680d1f90 0x7f08680c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:51.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.034+0000 7f086e096640 1 -- 172.21.15.69:0/2774468017 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08681fa0c0 con 0x7f08680b0730 2024-04-25T10:59:51.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.034+0000 7f086d094640 1 --2- 172.21.15.69:0/2774468017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08680d1f90 0x7f08680c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:51.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.034+0000 7f086e096640 1 --2- 172.21.15.69:0/2774468017 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08680b0730 0x7f086815ec20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0860007e60 tx=0x7f086000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:51.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.034+0000 7f085e7fc640 1 -- 172.21.15.69:0/2774468017 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0860014420 con 0x7f08680b0730 2024-04-25T10:59:51.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.034+0000 7f085e7fc640 1 -- 172.21.15.69:0/2774468017 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0860019070 con 0x7f08680b0730 2024-04-25T10:59:51.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.034+0000 7f086faf9640 1 -- 172.21.15.69:0/2774468017 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08681fa360 con 0x7f08680b0730 2024-04-25T10:59:51.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.034+0000 7f085e7fc640 1 -- 172.21.15.69:0/2774468017 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0860014820 con 0x7f08680b0730 2024-04-25T10:59:51.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.034+0000 7f086faf9640 1 -- 172.21.15.69:0/2774468017 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08681fa920 con 0x7f08680b0730 2024-04-25T10:59:51.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.035+0000 7f086faf9640 1 -- 172.21.15.69:0/2774468017 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0868050740 con 0x7f08680b0730 2024-04-25T10:59:51.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.036+0000 7f085e7fc640 1 -- 172.21.15.69:0/2774468017 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f08600149c0 con 0x7f08680b0730 2024-04-25T10:59:51.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.037+0000 7f085e7fc640 1 --2- 172.21.15.69:0/2774468017 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f084c0744e0 0x7f084c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:51.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.037+0000 7f086d895640 1 --2- 172.21.15.69:0/2774468017 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f084c0744e0 0x7f084c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:51.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.037+0000 7f085e7fc640 1 -- 172.21.15.69:0/2774468017 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0860095900 con 0x7f08680b0730 2024-04-25T10:59:51.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.037+0000 7f086d895640 1 --2- 172.21.15.69:0/2774468017 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f084c0744e0 0x7f084c0769f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0864034000 tx=0x7f086402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:51.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.041+0000 7f085e7fc640 1 -- 172.21.15.69:0/2774468017 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0860060120 con 0x7f08680b0730 2024-04-25T10:59:51.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.215+0000 7f086faf9640 1 -- 172.21.15.69:0/2774468017 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f08680cf560 con 0x7f08680b0730 2024-04-25T10:59:51.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.215+0000 7f085e7fc640 1 -- 172.21.15.69:0/2774468017 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f086005fa20 con 0x7f08680b0730 2024-04-25T10:59:51.216 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-25T10:59:51.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.217+0000 7f086faf9640 1 -- 172.21.15.69:0/2774468017 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f084c0744e0 msgr2=0x7f084c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:51.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.217+0000 7f086faf9640 1 --2- 172.21.15.69:0/2774468017 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f084c0744e0 0x7f084c0769f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0864034000 tx=0x7f086402e000 comp rx=0 tx=0).stop 2024-04-25T10:59:51.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.217+0000 7f086faf9640 1 -- 172.21.15.69:0/2774468017 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08680b0730 msgr2=0x7f086815ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:51.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.217+0000 7f086faf9640 1 --2- 172.21.15.69:0/2774468017 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08680b0730 0x7f086815ec20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0860007e60 tx=0x7f086000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:59:51.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.217+0000 7f086faf9640 1 -- 172.21.15.69:0/2774468017 shutdown_connections 2024-04-25T10:59:51.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.217+0000 7f086faf9640 1 --2- 172.21.15.69:0/2774468017 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f084c0744e0 0x7f084c0769f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:51.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.217+0000 7f086faf9640 1 --2- 172.21.15.69:0/2774468017 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f08680b0730 0x7f086815ec20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:51.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.217+0000 7f086faf9640 1 --2- 172.21.15.69:0/2774468017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08680d1f90 0x7f08680c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:51.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.217+0000 7f086faf9640 1 --2- 172.21.15.69:0/2774468017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08680d0190 0x7f08680b01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:51.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.217+0000 7f086faf9640 1 -- 172.21.15.69:0/2774468017 >> 172.21.15.69:0/2774468017 conn(0x7f086809b7d0 msgr2=0x7f08680b77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:51.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.217+0000 7f086faf9640 1 -- 172.21.15.69:0/2774468017 shutdown_connections 2024-04-25T10:59:51.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.217+0000 7f086faf9640 1 -- 172.21.15.69:0/2774468017 wait complete. 2024-04-25T10:59:51.229 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:59:51.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.336+0000 7fb82f642640 1 Processor -- start 2024-04-25T10:59:51.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.336+0000 7fb82f642640 1 -- start start 2024-04-25T10:59:51.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.337+0000 7fb82f642640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb8280d0e00 0x7fb8280d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:51.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.337+0000 7fb82f642640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8280d4a50 0x7fb8280d4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:51.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.337+0000 7fb82f642640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8280d1850 con 0x7fb82806c5c0 2024-04-25T10:59:51.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.337+0000 7fb82f642640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fb828160de0 con 0x7fb8280d0e00 2024-04-25T10:59:51.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.337+0000 7fb82f642640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb8281610c0 con 0x7fb8280d4a50 2024-04-25T10:59:51.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.337+0000 7fb82d3de640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb8280d0e00 0x7fb8280d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:51.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.337+0000 7fb82d3de640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb8280d0e00 0x7fb8280d1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55188/0 (socket says 172.21.15.69:55188) 2024-04-25T10:59:51.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.337+0000 7fb82d3de640 1 -- 172.21.15.69:0/3491607925 learned_addr learned my addr 172.21.15.69:0/3491607925 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:51.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.338+0000 7fb82d3de640 1 -- 172.21.15.69:0/3491607925 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8280d4a50 msgr2=0x7fb8280d4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:51.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.338+0000 7fb82d3de640 1 --2- 172.21.15.69:0/3491607925 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8280d4a50 0x7fb8280d4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:51.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.338+0000 7fb82d3de640 1 -- 172.21.15.69:0/3491607925 >> v1:172.21.15.146:6789/0 conn(0x7fb82806c5c0 legacy=0x7fb82806c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:51.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.338+0000 7fb82d3de640 1 -- 172.21.15.69:0/3491607925 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8281613a0 con 0x7fb8280d0e00 2024-04-25T10:59:51.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.338+0000 7fb82d3de640 1 --2- 172.21.15.69:0/3491607925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8280d0e00 0x7fb8280d1210 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb824009e50 tx=0x7fb82402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d49036823d97f3c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:51.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.338+0000 7fb81ffff640 1 -- 172.21.15.69:0/3491607925 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824036070 con 0x7fb8280d0e00 2024-04-25T10:59:51.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.338+0000 7fb81ffff640 1 -- 172.21.15.69:0/3491607925 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb824031600 con 0x7fb8280d0e00 2024-04-25T10:59:51.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.339+0000 7fb81ffff640 1 -- 172.21.15.69:0/3491607925 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8240319d0 con 0x7fb8280d0e00 2024-04-25T10:59:51.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.339+0000 7fb82f642640 1 -- 172.21.15.69:0/3491607925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8280d0e00 msgr2=0x7fb8280d1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:51.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.339+0000 7fb82f642640 1 --2- 172.21.15.69:0/3491607925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8280d0e00 0x7fb8280d1210 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb824009e50 tx=0x7fb82402a6c0 comp rx=0 tx=0).stop 2024-04-25T10:59:51.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.339+0000 7fb82f642640 1 -- 172.21.15.69:0/3491607925 shutdown_connections 2024-04-25T10:59:51.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.339+0000 7fb82f642640 1 --2- 172.21.15.69:0/3491607925 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8280d4a50 0x7fb8280d4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:51.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.339+0000 7fb82f642640 1 --2- 172.21.15.69:0/3491607925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8280d0e00 0x7fb8280d1210 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:51.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.339+0000 7fb82f642640 1 -- 172.21.15.69:0/3491607925 >> 172.21.15.69:0/3491607925 conn(0x7fb8280d5d90 msgr2=0x7fb8280d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:51.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.339+0000 7fb82f642640 1 -- 172.21.15.69:0/3491607925 shutdown_connections 2024-04-25T10:59:51.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.339+0000 7fb82f642640 1 -- 172.21.15.69:0/3491607925 wait complete. 2024-04-25T10:59:51.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.341+0000 7fb82f642640 1 Processor -- start 2024-04-25T10:59:51.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.341+0000 7fb82f642640 1 -- start start 2024-04-25T10:59:51.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.342+0000 7fb82f642640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb82806c5c0 0x7fb8280cc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:51.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.342+0000 7fb82f642640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8280d0e00 0x7fb82815c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:51.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.342+0000 7fb82f642640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8280d4a50 0x7fb8281ea820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:51.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.342+0000 7fb82f642640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb828164e00 con 0x7fb82806c5c0 2024-04-25T10:59:51.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.342+0000 7fb82f642640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb8280b5320 con 0x7fb8280d4a50 2024-04-25T10:59:51.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.342+0000 7fb82f642640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb828164f80 con 0x7fb8280d0e00 2024-04-25T10:59:51.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.342+0000 7fb82d3de640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb82806c5c0 0x7fb8280cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:51.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.342+0000 7fb82d3de640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb82806c5c0 0x7fb8280cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55194/0 (socket says 172.21.15.69:55194) 2024-04-25T10:59:51.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.342+0000 7fb82d3de640 1 -- 172.21.15.69:0/125995573 learned_addr learned my addr 172.21.15.69:0/125995573 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:51.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.343+0000 7fb82cbdd640 1 --2- 172.21.15.69:0/125995573 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8280d0e00 0x7fb82815c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:51.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.343+0000 7fb82dbdf640 1 --2- 172.21.15.69:0/125995573 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8280d4a50 0x7fb8281ea820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:51.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.343+0000 7fb82d3de640 1 -- 172.21.15.69:0/125995573 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8280d0e00 msgr2=0x7fb82815c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:51.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.343+0000 7fb82d3de640 1 --2- 172.21.15.69:0/125995573 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8280d0e00 0x7fb82815c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:51.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.343+0000 7fb82d3de640 1 -- 172.21.15.69:0/125995573 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8280d4a50 msgr2=0x7fb8281ea820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:51.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.343+0000 7fb82d3de640 1 --2- 172.21.15.69:0/125995573 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8280d4a50 0x7fb8281ea820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:51.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.343+0000 7fb82d3de640 1 -- 172.21.15.69:0/125995573 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb82815c840 con 0x7fb82806c5c0 2024-04-25T10:59:51.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.343+0000 7fb82dbdf640 1 --2- 172.21.15.69:0/125995573 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8280d4a50 0x7fb8281ea820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:51.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.343+0000 7fb82d3de640 1 --2- 172.21.15.69:0/125995573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb82806c5c0 0x7fb8280cc310 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb824009f70 tx=0x7fb824004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:51.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.343+0000 7fb81dffb640 1 -- 172.21.15.69:0/125995573 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824041020 con 0x7fb82806c5c0 2024-04-25T10:59:51.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.343+0000 7fb81dffb640 1 -- 172.21.15.69:0/125995573 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb824007800 con 0x7fb82806c5c0 2024-04-25T10:59:51.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.343+0000 7fb82f642640 1 -- 172.21.15.69:0/125995573 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8281ead60 con 0x7fb82806c5c0 2024-04-25T10:59:51.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.343+0000 7fb81dffb640 1 -- 172.21.15.69:0/125995573 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824036040 con 0x7fb82806c5c0 2024-04-25T10:59:51.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.343+0000 7fb82f642640 1 -- 172.21.15.69:0/125995573 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8280cb550 con 0x7fb82806c5c0 2024-04-25T10:59:51.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.345+0000 7fb82f642640 1 -- 172.21.15.69:0/125995573 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb828050740 con 0x7fb82806c5c0 2024-04-25T10:59:51.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.345+0000 7fb81dffb640 1 -- 172.21.15.69:0/125995573 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb824044020 con 0x7fb82806c5c0 2024-04-25T10:59:51.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.346+0000 7fb81dffb640 1 --2- 172.21.15.69:0/125995573 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb80c0744e0 0x7fb80c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:51.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.346+0000 7fb81dffb640 1 -- 172.21.15.69:0/125995573 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb8240b5e30 con 0x7fb82806c5c0 2024-04-25T10:59:51.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.346+0000 7fb82cbdd640 1 --2- 172.21.15.69:0/125995573 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb80c0744e0 0x7fb80c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:51.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.346+0000 7fb82cbdd640 1 --2- 172.21.15.69:0/125995573 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb80c0744e0 0x7fb80c0769f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb818006fd0 tx=0x7fb818008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:51.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.350+0000 7fb81dffb640 1 -- 172.21.15.69:0/125995573 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb82404b070 con 0x7fb82806c5c0 2024-04-25T10:59:51.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.537+0000 7fb82f642640 1 -- 172.21.15.69:0/125995573 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb8280d4320 con 0x7fb82806c5c0 2024-04-25T10:59:51.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.538+0000 7fb81dffb640 1 -- 172.21.15.69:0/125995573 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7fb82815c840 con 0x7fb82806c5c0 2024-04-25T10:59:51.538 INFO:teuthology.orchestra.run.smithi069.stderr:listed 0 entries 2024-04-25T10:59:51.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.540+0000 7fb82f642640 1 -- 172.21.15.69:0/125995573 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb80c0744e0 msgr2=0x7fb80c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:51.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.540+0000 7fb82f642640 1 --2- 172.21.15.69:0/125995573 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb80c0744e0 0x7fb80c0769f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb818006fd0 tx=0x7fb818008040 comp rx=0 tx=0).stop 2024-04-25T10:59:51.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.540+0000 7fb82f642640 1 -- 172.21.15.69:0/125995573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb82806c5c0 msgr2=0x7fb8280cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:51.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.540+0000 7fb82f642640 1 --2- 172.21.15.69:0/125995573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb82806c5c0 0x7fb8280cc310 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb824009f70 tx=0x7fb824004a40 comp rx=0 tx=0).stop 2024-04-25T10:59:51.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.540+0000 7fb82f642640 1 -- 172.21.15.69:0/125995573 shutdown_connections 2024-04-25T10:59:51.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.540+0000 7fb82f642640 1 --2- 172.21.15.69:0/125995573 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb80c0744e0 0x7fb80c0769f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:51.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.540+0000 7fb82f642640 1 --2- 172.21.15.69:0/125995573 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8280d4a50 0x7fb8281ea820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:51.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.540+0000 7fb82f642640 1 --2- 172.21.15.69:0/125995573 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8280d0e00 0x7fb82815c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:51.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.540+0000 7fb82f642640 1 --2- 172.21.15.69:0/125995573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb82806c5c0 0x7fb8280cc310 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:51.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.540+0000 7fb82f642640 1 -- 172.21.15.69:0/125995573 >> 172.21.15.69:0/125995573 conn(0x7fb8280d5d90 msgr2=0x7fb8280d4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:51.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.540+0000 7fb82f642640 1 -- 172.21.15.69:0/125995573 shutdown_connections 2024-04-25T10:59:51.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.540+0000 7fb82f642640 1 -- 172.21.15.69:0/125995573 wait complete. 2024-04-25T10:59:51.552 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-25T10:59:51.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:51.552 DEBUG:teuthology.orchestra.run.smithi069:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-25T10:59:51.576 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi069.front.sepia.ceph.com 2024-04-25T10:59:51.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:51.577 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:59:51.631 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T10:59:51.632 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:59:51.652 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T10:59:51.652 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:59:51.712 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T10:59:51.712 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:59:51.771 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:59:51.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.879+0000 7fe0b3240640 1 Processor -- start 2024-04-25T10:59:51.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.879+0000 7fe0b3240640 1 -- start start 2024-04-25T10:59:51.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.880+0000 7fe0b3240640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe0ac0d0e00 0x7fe0ac0d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:51.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.880+0000 7fe0b3240640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe0ac0d4a50 0x7fe0ac0d4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:51.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.880+0000 7fe0b3240640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0ac0d1850 con 0x7fe0ac06c5c0 2024-04-25T10:59:51.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.880+0000 7fe0b3240640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fe0ac160de0 con 0x7fe0ac0d0e00 2024-04-25T10:59:51.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.880+0000 7fe0b3240640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fe0ac1610c0 con 0x7fe0ac0d4a50 2024-04-25T10:59:51.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.880+0000 7fe0b0fdc640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe0ac0d0e00 0x7fe0ac0d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:51.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.880+0000 7fe0b0fdc640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe0ac0d0e00 0x7fe0ac0d1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55208/0 (socket says 172.21.15.69:55208) 2024-04-25T10:59:51.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.880+0000 7fe0abfff640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fe0ac06c5c0 0x7fe0ac06c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:50184/0 (socket says 172.21.15.69:50184) 2024-04-25T10:59:51.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.880+0000 7fe0b0fdc640 1 -- 172.21.15.69:0/3724408325 learned_addr learned my addr 172.21.15.69:0/3724408325 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:51.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.880+0000 7fe0b0fdc640 1 -- 172.21.15.69:0/3724408325 >> v2:172.21.15.146:3300/0 conn(0x7fe0ac0d4a50 msgr2=0x7fe0ac0d4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:51.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.880+0000 7fe0b17dd640 1 --2- 172.21.15.69:0/3724408325 >> v2:172.21.15.146:3300/0 conn(0x7fe0ac0d4a50 0x7fe0ac0d4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:51.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.880+0000 7fe0b0fdc640 1 --2- 172.21.15.69:0/3724408325 >> v2:172.21.15.146:3300/0 conn(0x7fe0ac0d4a50 0x7fe0ac0d4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:51.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.880+0000 7fe0ab7fe640 1 -- 172.21.15.69:0/3724408325 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3990786769 0 0) 0x7fe0ac0d1850 con 0x7fe0ac06c5c0 2024-04-25T10:59:51.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.881+0000 7fe0b0fdc640 1 -- 172.21.15.69:0/3724408325 >> v1:172.21.15.69:6789/0 conn(0x7fe0ac06c5c0 legacy=0x7fe0ac06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:51.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.881+0000 7fe0b0fdc640 1 -- 172.21.15.69:0/3724408325 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0ac1613a0 con 0x7fe0ac0d0e00 2024-04-25T10:59:51.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.881+0000 7fe0b17dd640 1 --2- 172.21.15.69:0/3724408325 >> v2:172.21.15.146:3300/0 conn(0x7fe0ac0d4a50 0x7fe0ac0d4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:51.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.881+0000 7fe0b0fdc640 1 --2- 172.21.15.69:0/3724408325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0ac0d0e00 0x7fe0ac0d1210 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac0d1750 tx=0x7fe09c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f046fd7dc8408906 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:51.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.881+0000 7fe0ab7fe640 1 -- 172.21.15.69:0/3724408325 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09c037070 con 0x7fe0ac0d0e00 2024-04-25T10:59:51.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.881+0000 7fe0ab7fe640 1 -- 172.21.15.69:0/3724408325 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe09c032440 con 0x7fe0ac0d0e00 2024-04-25T10:59:51.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.881+0000 7fe0ab7fe640 1 -- 172.21.15.69:0/3724408325 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09c032810 con 0x7fe0ac0d0e00 2024-04-25T10:59:51.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.882+0000 7fe0b3240640 1 -- 172.21.15.69:0/3724408325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0ac0d0e00 msgr2=0x7fe0ac0d1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:51.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.882+0000 7fe0b3240640 1 --2- 172.21.15.69:0/3724408325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0ac0d0e00 0x7fe0ac0d1210 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac0d1750 tx=0x7fe09c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:59:51.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.882+0000 7fe0b3240640 1 -- 172.21.15.69:0/3724408325 shutdown_connections 2024-04-25T10:59:51.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.882+0000 7fe0b3240640 1 --2- 172.21.15.69:0/3724408325 >> v2:172.21.15.146:3300/0 conn(0x7fe0ac0d4a50 0x7fe0ac0d4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:51.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.882+0000 7fe0b3240640 1 --2- 172.21.15.69:0/3724408325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0ac0d0e00 0x7fe0ac0d1210 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:51.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.882+0000 7fe0b3240640 1 -- 172.21.15.69:0/3724408325 >> 172.21.15.69:0/3724408325 conn(0x7fe0ac0d5d90 msgr2=0x7fe0ac0d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:51.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.882+0000 7fe0b3240640 1 -- 172.21.15.69:0/3724408325 shutdown_connections 2024-04-25T10:59:51.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.882+0000 7fe0b3240640 1 -- 172.21.15.69:0/3724408325 wait complete. 2024-04-25T10:59:51.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.884+0000 7fe0b3240640 1 Processor -- start 2024-04-25T10:59:51.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.884+0000 7fe0b3240640 1 -- start start 2024-04-25T10:59:51.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.885+0000 7fe0b3240640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0ac06c5c0 0x7fe0ac0c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:51.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.885+0000 7fe0b3240640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0ac0d0e00 0x7fe0ac0cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:51.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.885+0000 7fe0b3240640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe0ac0d4a50 0x7fe0ac14f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:51.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.885+0000 7fe0b3240640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac164e00 con 0x7fe0ac0d0e00 2024-04-25T10:59:51.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.885+0000 7fe0b3240640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac0b5320 con 0x7fe0ac06c5c0 2024-04-25T10:59:51.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.885+0000 7fe0b3240640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe0ac164f80 con 0x7fe0ac0d4a50 2024-04-25T10:59:51.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.885+0000 7fe0abfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0ac0d0e00 0x7fe0ac0cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:51.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.885+0000 7fe0abfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0ac0d0e00 0x7fe0ac0cc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55224/0 (socket says 172.21.15.69:55224) 2024-04-25T10:59:51.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.885+0000 7fe0abfff640 1 -- 172.21.15.69:0/2521286575 learned_addr learned my addr 172.21.15.69:0/2521286575 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:51.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.886+0000 7fe0b0fdc640 1 --2- 172.21.15.69:0/2521286575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0ac06c5c0 0x7fe0ac0c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:51.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.886+0000 7fe0b17dd640 1 --2- 172.21.15.69:0/2521286575 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe0ac0d4a50 0x7fe0ac14f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:51.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.886+0000 7fe0abfff640 1 -- 172.21.15.69:0/2521286575 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe0ac0d4a50 msgr2=0x7fe0ac14f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:51.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.886+0000 7fe0abfff640 1 --2- 172.21.15.69:0/2521286575 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe0ac0d4a50 0x7fe0ac14f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:51.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.886+0000 7fe0abfff640 1 -- 172.21.15.69:0/2521286575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0ac06c5c0 msgr2=0x7fe0ac0c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:51.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.886+0000 7fe0abfff640 1 --2- 172.21.15.69:0/2521286575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0ac06c5c0 0x7fe0ac0c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:51.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.886+0000 7fe0abfff640 1 -- 172.21.15.69:0/2521286575 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0ac14d630 con 0x7fe0ac0d0e00 2024-04-25T10:59:51.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.886+0000 7fe0b17dd640 1 --2- 172.21.15.69:0/2521286575 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe0ac0d4a50 0x7fe0ac14f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:51.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.886+0000 7fe0abfff640 1 --2- 172.21.15.69:0/2521286575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0ac0d0e00 0x7fe0ac0cc3a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe0a000c430 tx=0x7fe0a000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:51.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.886+0000 7fe0a97fa640 1 -- 172.21.15.69:0/2521286575 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a00074e0 con 0x7fe0ac0d0e00 2024-04-25T10:59:51.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.886+0000 7fe0a97fa640 1 -- 172.21.15.69:0/2521286575 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0a00047e0 con 0x7fe0ac0d0e00 2024-04-25T10:59:51.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.886+0000 7fe0b3240640 1 -- 172.21.15.69:0/2521286575 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0ac14d910 con 0x7fe0ac0d0e00 2024-04-25T10:59:51.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.886+0000 7fe0a97fa640 1 -- 172.21.15.69:0/2521286575 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0004bd0 con 0x7fe0ac0d0e00 2024-04-25T10:59:51.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.886+0000 7fe0b3240640 1 -- 172.21.15.69:0/2521286575 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0ac14ded0 con 0x7fe0ac0d0e00 2024-04-25T10:59:51.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.887+0000 7fe0b3240640 1 -- 172.21.15.69:0/2521286575 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0ac050740 con 0x7fe0ac0d0e00 2024-04-25T10:59:51.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.888+0000 7fe0a97fa640 1 -- 172.21.15.69:0/2521286575 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe0a0011070 con 0x7fe0ac0d0e00 2024-04-25T10:59:51.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.889+0000 7fe0a97fa640 1 --2- 172.21.15.69:0/2521286575 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe0980744b0 0x7fe0980769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:51.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.889+0000 7fe0b0fdc640 1 --2- 172.21.15.69:0/2521286575 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe0980744b0 0x7fe0980769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:51.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.889+0000 7fe0a97fa640 1 -- 172.21.15.69:0/2521286575 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe0a0094d10 con 0x7fe0ac0d0e00 2024-04-25T10:59:51.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.889+0000 7fe0b0fdc640 1 --2- 172.21.15.69:0/2521286575 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe0980744b0 0x7fe0980769c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe09c034000 tx=0x7fe09c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:51.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:51.893+0000 7fe0a97fa640 1 -- 172.21.15.69:0/2521286575 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe0a005f5c0 con 0x7fe0ac0d0e00 2024-04-25T10:59:52.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.073+0000 7fe0b3240640 1 -- 172.21.15.69:0/2521286575 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe0ac160840 con 0x7fe0ac0d0e00 2024-04-25T10:59:52.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.074+0000 7fe0a97fa640 1 -- 172.21.15.69:0/2521286575 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7fe0a005eec0 con 0x7fe0ac0d0e00 2024-04-25T10:59:52.075 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:52.075 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":20,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T10:59:20.630151+0000","last_up_change":"2024-04-25T10:59:05.546839+0000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:59:06.422674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:59:16.727980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:59:17.938486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6825","nonce":1876101065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6827","nonce":1876101065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6831","nonce":1876101065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6829","nonce":1876101065}]},"public_addr":"172.21.15.69:6825/1876101065","cluster_addr":"172.21.15.69:6827/1876101065","heartbeat_back_addr":"172.21.15.69:6831/1876101065","heartbeat_front_addr":"172.21.15.69:6829/1876101065","state":["exists","up"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6801","nonce":1806438563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6803","nonce":1806438563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6807","nonce":1806438563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6805","nonce":1806438563}]},"public_addr":"172.21.15.69:6801/1806438563","cluster_addr":"172.21.15.69:6803/1806438563","heartbeat_back_addr":"172.21.15.69:6807/1806438563","heartbeat_front_addr":"172.21.15.69:6805/1806438563","state":["exists","up"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6817","nonce":1593617983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6819","nonce":1593617983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6823","nonce":1593617983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6821","nonce":1593617983}]},"public_addr":"172.21.15.69:6817/1593617983","cluster_addr":"172.21.15.69:6819/1593617983","heartbeat_back_addr":"172.21.15.69:6823/1593617983","heartbeat_front_addr":"172.21.15.69:6821/1593617983","state":["exists","up"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6809","nonce":3950661294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6811","nonce":3950661294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6815","nonce":3950661294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6813","nonce":3950661294}]},"public_addr":"172.21.15.69:6809/3950661294","cluster_addr":"172.21.15.69:6811/3950661294","heartbeat_back_addr":"172.21.15.69:6815/3950661294","heartbeat_front_addr":"172.21.15.69:6813/3950661294","state":["exists","up"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6801","nonce":2508929872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6803","nonce":2508929872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6807","nonce":2508929872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6805","nonce":2508929872}]},"public_addr":"172.21.15.146:6801/2508929872","cluster_addr":"172.21.15.146:6803/2508929872","heartbeat_back_addr":"172.21.15.146:6807/2508929872","heartbeat_front_addr":"172.21.15.146:6805/2508929872","state":["exists","up"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6811","nonce":1310911138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6816","nonce":1310911138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6823","nonce":1310911138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6821","nonce":1310911138}]},"public_addr":"172.21.15.146:6811/1310911138","cluster_addr":"172.21.15.146:6816/1310911138","heartbeat_back_addr":"172.21.15.146:6823/1310911138","heartbeat_front_addr":"172.21.15.146:6821/1310911138","state":["exists","up"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6810","nonce":634688021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6813","nonce":634688021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6820","nonce":634688021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6817","nonce":634688021}]},"public_addr":"172.21.15.146:6810/634688021","cluster_addr":"172.21.15.146:6813/634688021","heartbeat_back_addr":"172.21.15.146:6820/634688021","heartbeat_front_addr":"172.21.15.146:6817/634688021","state":["exists","up"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6825","nonce":3814215935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6827","nonce":3814215935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6831","nonce":3814215935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6829","nonce":3814215935}]},"public_addr":"172.21.15.146:6825/3814215935","cluster_addr":"172.21.15.146:6827/3814215935","heartbeat_back_addr":"172.21.15.146:6831/3814215935","heartbeat_front_addr":"172.21.15.146:6829/3814215935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.820200+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.850189+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.764560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.866248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.807897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.869881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.782310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.827131+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:59:52.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.076+0000 7fe0b3240640 1 -- 172.21.15.69:0/2521286575 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe0980744b0 msgr2=0x7fe0980769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:52.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.076+0000 7fe0b3240640 1 --2- 172.21.15.69:0/2521286575 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe0980744b0 0x7fe0980769c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe09c034000 tx=0x7fe09c02e000 comp rx=0 tx=0).stop 2024-04-25T10:59:52.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.077+0000 7fe0b3240640 1 -- 172.21.15.69:0/2521286575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0ac0d0e00 msgr2=0x7fe0ac0cc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:52.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.077+0000 7fe0b3240640 1 --2- 172.21.15.69:0/2521286575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0ac0d0e00 0x7fe0ac0cc3a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe0a000c430 tx=0x7fe0a000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:59:52.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.077+0000 7fe0b3240640 1 -- 172.21.15.69:0/2521286575 shutdown_connections 2024-04-25T10:59:52.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.077+0000 7fe0b3240640 1 --2- 172.21.15.69:0/2521286575 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe0980744b0 0x7fe0980769c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:52.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.077+0000 7fe0b3240640 1 --2- 172.21.15.69:0/2521286575 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe0ac0d4a50 0x7fe0ac14f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:52.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.077+0000 7fe0b3240640 1 --2- 172.21.15.69:0/2521286575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0ac0d0e00 0x7fe0ac0cc3a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:52.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.077+0000 7fe0b3240640 1 --2- 172.21.15.69:0/2521286575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0ac06c5c0 0x7fe0ac0c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:52.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.077+0000 7fe0b3240640 1 -- 172.21.15.69:0/2521286575 >> 172.21.15.69:0/2521286575 conn(0x7fe0ac0d5d90 msgr2=0x7fe0ac0b58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:52.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.077+0000 7fe0b3240640 1 -- 172.21.15.69:0/2521286575 shutdown_connections 2024-04-25T10:59:52.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.077+0000 7fe0b3240640 1 -- 172.21.15.69:0/2521286575 wait complete. 2024-04-25T10:59:52.089 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:59:06.422674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:59:16.727980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:59:17.938486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:59:52.089 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:59:52.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.198+0000 7faa093ee640 1 Processor -- start 2024-04-25T10:59:52.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.198+0000 7faa093ee640 1 -- start start 2024-04-25T10:59:52.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.200+0000 7faa093ee640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7faa04161710 0x7faa0415e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:52.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.200+0000 7faa093ee640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa041609b0 con 0x7faa04163fa0 2024-04-25T10:59:52.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.200+0000 7faa093ee640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa04164cd0 con 0x7faa0406c5c0 2024-04-25T10:59:52.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.200+0000 7faa093ee640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7faa04160c30 con 0x7faa04161710 2024-04-25T10:59:52.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.200+0000 7faa037fe640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7faa04163fa0 0x7faa041643b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:50190/0 (socket says 172.21.15.69:50190) 2024-04-25T10:59:52.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.200+0000 7faa037fe640 1 -- 172.21.15.69:0/3559220199 learned_addr learned my addr 172.21.15.69:0/3559220199 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:52.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.200+0000 7faa02ffd640 1 --2- 172.21.15.69:0/3559220199 >> v2:172.21.15.146:3300/0 conn(0x7faa04161710 0x7faa0415e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:52.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.200+0000 7faa01ffb640 1 -- 172.21.15.69:0/3559220199 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1960165664 0 0) 0x7fa9fc001110 con 0x7faa04163fa0 2024-04-25T10:59:52.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.200+0000 7faa01ffb640 1 -- 172.21.15.69:0/3559220199 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9ec003520 con 0x7faa04163fa0 2024-04-25T10:59:52.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.200+0000 7faa02ffd640 1 -- 172.21.15.69:0/3559220199 >> v1:172.21.15.146:6789/0 conn(0x7faa0406c5c0 legacy=0x7faa0406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:52.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.200+0000 7faa01ffb640 1 -- 172.21.15.69:0/3559220199 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 790400442 0 0) 0x7fa9ec003520 con 0x7faa04163fa0 2024-04-25T10:59:52.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.200+0000 7faa02ffd640 1 -- 172.21.15.69:0/3559220199 >> v1:172.21.15.69:6789/0 conn(0x7faa04163fa0 legacy=0x7faa041643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:52.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.200+0000 7faa02ffd640 1 -- 172.21.15.69:0/3559220199 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa04160f10 con 0x7faa04161710 2024-04-25T10:59:52.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.201+0000 7faa02ffd640 1 --2- 172.21.15.69:0/3559220199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa04161710 0x7faa0415e080 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4009d90 tx=0x7fa9f402a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85f48de25d4dfe46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:52.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.201+0000 7faa01ffb640 1 -- 172.21.15.69:0/3559220199 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc001840 con 0x7faa04161710 2024-04-25T10:59:52.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.201+0000 7faa01ffb640 1 -- 172.21.15.69:0/3559220199 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9f4007540 con 0x7faa04161710 2024-04-25T10:59:52.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.201+0000 7faa01ffb640 1 -- 172.21.15.69:0/3559220199 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f40078f0 con 0x7faa04161710 2024-04-25T10:59:52.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.202+0000 7faa093ee640 1 -- 172.21.15.69:0/3559220199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa04161710 msgr2=0x7faa0415e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:52.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.202+0000 7faa093ee640 1 --2- 172.21.15.69:0/3559220199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa04161710 0x7faa0415e080 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4009d90 tx=0x7fa9f402a5f0 comp rx=0 tx=0).stop 2024-04-25T10:59:52.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.202+0000 7faa093ee640 1 -- 172.21.15.69:0/3559220199 shutdown_connections 2024-04-25T10:59:52.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.202+0000 7faa093ee640 1 --2- 172.21.15.69:0/3559220199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa04161710 0x7faa0415e080 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:52.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.202+0000 7faa093ee640 1 -- 172.21.15.69:0/3559220199 >> 172.21.15.69:0/3559220199 conn(0x7faa040d5d90 msgr2=0x7faa040d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:52.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.202+0000 7faa093ee640 1 -- 172.21.15.69:0/3559220199 shutdown_connections 2024-04-25T10:59:52.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.202+0000 7faa093ee640 1 -- 172.21.15.69:0/3559220199 wait complete. 2024-04-25T10:59:52.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.204+0000 7faa093ee640 1 Processor -- start 2024-04-25T10:59:52.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.204+0000 7faa093ee640 1 -- start start 2024-04-25T10:59:52.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.205+0000 7faa093ee640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa0406c5c0 0x7faa0414fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:52.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.205+0000 7faa093ee640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa04161710 0x7faa040b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:52.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.205+0000 7faa093ee640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa04163fa0 0x7faa04154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:52.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.205+0000 7faa093ee640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7faa0416b280 con 0x7faa04163fa0 2024-04-25T10:59:52.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.205+0000 7faa093ee640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7faa0416b100 con 0x7faa04161710 2024-04-25T10:59:52.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.205+0000 7faa093ee640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7faa0416b400 con 0x7faa0406c5c0 2024-04-25T10:59:52.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.205+0000 7faa037fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa04163fa0 0x7faa04154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:52.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.205+0000 7faa037fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa04163fa0 0x7faa04154490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55228/0 (socket says 172.21.15.69:55228) 2024-04-25T10:59:52.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.205+0000 7faa037fe640 1 -- 172.21.15.69:0/2493661048 learned_addr learned my addr 172.21.15.69:0/2493661048 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:52.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.206+0000 7faa02ffd640 1 --2- 172.21.15.69:0/2493661048 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa0406c5c0 0x7faa0414fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:52.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.206+0000 7faa027fc640 1 --2- 172.21.15.69:0/2493661048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa04161710 0x7faa040b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:52.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.206+0000 7faa037fe640 1 -- 172.21.15.69:0/2493661048 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa0406c5c0 msgr2=0x7faa0414fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:52.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.206+0000 7faa037fe640 1 --2- 172.21.15.69:0/2493661048 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa0406c5c0 0x7faa0414fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:52.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.206+0000 7faa037fe640 1 -- 172.21.15.69:0/2493661048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa04161710 msgr2=0x7faa040b7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:52.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.206+0000 7faa037fe640 1 --2- 172.21.15.69:0/2493661048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa04161710 0x7faa040b7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:52.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.206+0000 7faa037fe640 1 -- 172.21.15.69:0/2493661048 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa04150400 con 0x7faa04163fa0 2024-04-25T10:59:52.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.206+0000 7faa027fc640 1 --2- 172.21.15.69:0/2493661048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa04161710 0x7faa040b7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:52.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.206+0000 7faa037fe640 1 --2- 172.21.15.69:0/2493661048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa04163fa0 0x7faa04154490 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc00c4b0 tx=0x7fa9fc00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:52.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.206+0000 7fa9eb7fe640 1 -- 172.21.15.69:0/2493661048 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc017070 con 0x7faa04163fa0 2024-04-25T10:59:52.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.206+0000 7fa9eb7fe640 1 -- 172.21.15.69:0/2493661048 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9fc005ce0 con 0x7faa04163fa0 2024-04-25T10:59:52.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.206+0000 7fa9eb7fe640 1 -- 172.21.15.69:0/2493661048 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc007290 con 0x7faa04163fa0 2024-04-25T10:59:52.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.206+0000 7faa093ee640 1 -- 172.21.15.69:0/2493661048 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa04150910 con 0x7faa04163fa0 2024-04-25T10:59:52.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.206+0000 7faa093ee640 1 -- 172.21.15.69:0/2493661048 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa04150ed0 con 0x7faa04163fa0 2024-04-25T10:59:52.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.207+0000 7faa093ee640 1 -- 172.21.15.69:0/2493661048 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa04050740 con 0x7faa04163fa0 2024-04-25T10:59:52.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.208+0000 7fa9eb7fe640 1 -- 172.21.15.69:0/2493661048 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa9fc01a070 con 0x7faa04163fa0 2024-04-25T10:59:52.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.208+0000 7fa9eb7fe640 1 --2- 172.21.15.69:0/2493661048 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa9f00744e0 0x7fa9f00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:52.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.208+0000 7fa9eb7fe640 1 -- 172.21.15.69:0/2493661048 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa9fc094620 con 0x7faa04163fa0 2024-04-25T10:59:52.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.208+0000 7faa02ffd640 1 --2- 172.21.15.69:0/2493661048 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa9f00744e0 0x7fa9f00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:52.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.209+0000 7faa02ffd640 1 --2- 172.21.15.69:0/2493661048 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa9f00744e0 0x7fa9f00769f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4034000 tx=0x7fa9f402f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:52.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.213+0000 7fa9eb7fe640 1 -- 172.21.15.69:0/2493661048 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faa041609b0 con 0x7faa04163fa0 2024-04-25T10:59:52.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.386+0000 7faa093ee640 1 -- 172.21.15.69:0/2493661048 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faa040d12f0 con 0x7faa04163fa0 2024-04-25T10:59:52.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.386+0000 7fa9eb7fe640 1 -- 172.21.15.69:0/2493661048 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fa9fc05edb0 con 0x7faa04163fa0 2024-04-25T10:59:52.387 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-25T10:59:52.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.388+0000 7faa093ee640 1 -- 172.21.15.69:0/2493661048 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa9f00744e0 msgr2=0x7fa9f00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:52.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.388+0000 7faa093ee640 1 --2- 172.21.15.69:0/2493661048 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa9f00744e0 0x7fa9f00769f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4034000 tx=0x7fa9f402f000 comp rx=0 tx=0).stop 2024-04-25T10:59:52.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.388+0000 7faa093ee640 1 -- 172.21.15.69:0/2493661048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa04163fa0 msgr2=0x7faa04154490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:52.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.388+0000 7faa093ee640 1 --2- 172.21.15.69:0/2493661048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa04163fa0 0x7faa04154490 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc00c4b0 tx=0x7fa9fc00c970 comp rx=0 tx=0).stop 2024-04-25T10:59:52.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.388+0000 7faa093ee640 1 -- 172.21.15.69:0/2493661048 shutdown_connections 2024-04-25T10:59:52.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.388+0000 7faa093ee640 1 --2- 172.21.15.69:0/2493661048 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa9f00744e0 0x7fa9f00769f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:52.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.388+0000 7faa093ee640 1 --2- 172.21.15.69:0/2493661048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa04163fa0 0x7faa04154490 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:52.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.388+0000 7faa093ee640 1 --2- 172.21.15.69:0/2493661048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa04161710 0x7faa040b7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:52.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.388+0000 7faa093ee640 1 --2- 172.21.15.69:0/2493661048 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa0406c5c0 0x7faa0414fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:52.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.388+0000 7faa093ee640 1 -- 172.21.15.69:0/2493661048 >> 172.21.15.69:0/2493661048 conn(0x7faa040d5d90 msgr2=0x7faa04155890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:52.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.388+0000 7faa093ee640 1 -- 172.21.15.69:0/2493661048 shutdown_connections 2024-04-25T10:59:52.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.388+0000 7faa093ee640 1 -- 172.21.15.69:0/2493661048 wait complete. 2024-04-25T10:59:52.400 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:59:52.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.509+0000 7f3fd7f96640 1 Processor -- start 2024-04-25T10:59:52.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.509+0000 7f3fd7f96640 1 -- start start 2024-04-25T10:59:52.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.510+0000 7f3fd7f96640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3fd00d1e20 0x7f3fd00d3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:52.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.510+0000 7f3fd7f96640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3fd00b2e60 0x7f3fd008d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:52.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.510+0000 7f3fd7f96640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fd00d23f0 con 0x7f3fd0164020 2024-04-25T10:59:52.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.510+0000 7f3fd7f96640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f3fd00d26d0 con 0x7f3fd00b2e60 2024-04-25T10:59:52.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.510+0000 7f3fd7f96640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3fd00d29b0 con 0x7f3fd00d1e20 2024-04-25T10:59:52.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.510+0000 7f3fd6533640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f3fd0164020 0x7f3fd0164430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:50202/0 (socket says 172.21.15.69:50202) 2024-04-25T10:59:52.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.510+0000 7f3fd6533640 1 -- 172.21.15.69:0/2220509413 learned_addr learned my addr 172.21.15.69:0/2220509413 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:52.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.511+0000 7f3fd5531640 1 --2- 172.21.15.69:0/2220509413 >> v2:172.21.15.146:3300/0 conn(0x7f3fd00b2e60 0x7f3fd008d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:52.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.511+0000 7f3fd5d32640 1 --2- 172.21.15.69:0/2220509413 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3fd00d1e20 0x7f3fd00d3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:52.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.511+0000 7f3fd4d30640 1 -- 172.21.15.69:0/2220509413 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1545185703 0 0) 0x7f3fd00d23f0 con 0x7f3fd0164020 2024-04-25T10:59:52.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.511+0000 7f3fd4d30640 1 -- 172.21.15.69:0/2220509413 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3fbc003520 con 0x7f3fd0164020 2024-04-25T10:59:52.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.511+0000 7f3fd5531640 1 -- 172.21.15.69:0/2220509413 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3fd00d1e20 msgr2=0x7f3fd00d3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:52.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.511+0000 7f3fd5531640 1 --2- 172.21.15.69:0/2220509413 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3fd00d1e20 0x7f3fd00d3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:52.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.511+0000 7f3fd4d30640 1 -- 172.21.15.69:0/2220509413 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 377285615 0 0) 0x7f3fbc003520 con 0x7f3fd0164020 2024-04-25T10:59:52.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.511+0000 7f3fd5531640 1 -- 172.21.15.69:0/2220509413 >> v1:172.21.15.69:6789/0 conn(0x7f3fd0164020 legacy=0x7f3fd0164430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:52.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.511+0000 7f3fd5531640 1 -- 172.21.15.69:0/2220509413 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fd00d2c90 con 0x7f3fd00b2e60 2024-04-25T10:59:52.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.512+0000 7f3fd5531640 1 --2- 172.21.15.69:0/2220509413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fd00b2e60 0x7f3fd008d770 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00097a0 tx=0x7f3fc002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5b704fd0e779f4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:52.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.512+0000 7f3fd4d30640 1 -- 172.21.15.69:0/2220509413 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0037070 con 0x7f3fd00b2e60 2024-04-25T10:59:52.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.512+0000 7f3fd4d30640 1 -- 172.21.15.69:0/2220509413 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fc002ac30 con 0x7f3fd00b2e60 2024-04-25T10:59:52.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.512+0000 7f3fd4d30640 1 -- 172.21.15.69:0/2220509413 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc80019f0 con 0x7f3fd00b2e60 2024-04-25T10:59:52.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.512+0000 7f3fd7f96640 1 -- 172.21.15.69:0/2220509413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fd00b2e60 msgr2=0x7f3fd008d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:52.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.512+0000 7f3fd7f96640 1 --2- 172.21.15.69:0/2220509413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fd00b2e60 0x7f3fd008d770 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00097a0 tx=0x7f3fc002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:59:52.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.513+0000 7f3fd7f96640 1 -- 172.21.15.69:0/2220509413 shutdown_connections 2024-04-25T10:59:52.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.513+0000 7f3fd7f96640 1 --2- 172.21.15.69:0/2220509413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fd00b2e60 0x7f3fd008d770 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:52.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.513+0000 7f3fd7f96640 1 --2- 172.21.15.69:0/2220509413 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3fd00d1e20 0x7f3fd00d3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:52.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.513+0000 7f3fd7f96640 1 -- 172.21.15.69:0/2220509413 >> 172.21.15.69:0/2220509413 conn(0x7f3fd00d0530 msgr2=0x7f3fd00d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:52.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.513+0000 7f3fd7f96640 1 -- 172.21.15.69:0/2220509413 shutdown_connections 2024-04-25T10:59:52.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.513+0000 7f3fd7f96640 1 -- 172.21.15.69:0/2220509413 wait complete. 2024-04-25T10:59:52.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.514+0000 7f3fd7f96640 1 Processor -- start 2024-04-25T10:59:52.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.514+0000 7f3fd7f96640 1 -- start start 2024-04-25T10:59:52.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.516+0000 7f3fd7f96640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fd00b2e60 0x7f3fd014df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:52.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.516+0000 7f3fd7f96640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3fd00d1e20 0x7f3fd00cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:52.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.516+0000 7f3fd7f96640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fd0164020 0x7f3fd00ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:52.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.516+0000 7f3fd7f96640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd00ce070 con 0x7f3fd00b2e60 2024-04-25T10:59:52.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.516+0000 7f3fd7f96640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd008dcb0 con 0x7f3fd0164020 2024-04-25T10:59:52.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.516+0000 7f3fd7f96640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3fd00ce1f0 con 0x7f3fd00d1e20 2024-04-25T10:59:52.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.516+0000 7f3fd5d32640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fd00b2e60 0x7f3fd014df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:52.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.516+0000 7f3fd5d32640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fd00b2e60 0x7f3fd014df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55238/0 (socket says 172.21.15.69:55238) 2024-04-25T10:59:52.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.516+0000 7f3fd5d32640 1 -- 172.21.15.69:0/338726576 learned_addr learned my addr 172.21.15.69:0/338726576 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:52.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.516+0000 7f3fd5531640 1 --2- 172.21.15.69:0/338726576 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3fd00d1e20 0x7f3fd00cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:52.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.516+0000 7f3fd6533640 1 --2- 172.21.15.69:0/338726576 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fd0164020 0x7f3fd00ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:52.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.516+0000 7f3fd5d32640 1 -- 172.21.15.69:0/338726576 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3fd00d1e20 msgr2=0x7f3fd00cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:52.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.516+0000 7f3fd5d32640 1 --2- 172.21.15.69:0/338726576 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3fd00d1e20 0x7f3fd00cc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:52.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.516+0000 7f3fd5d32640 1 -- 172.21.15.69:0/338726576 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fd0164020 msgr2=0x7f3fd00ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:52.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.516+0000 7f3fd5d32640 1 --2- 172.21.15.69:0/338726576 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fd0164020 0x7f3fd00ccc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:52.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.516+0000 7f3fd5d32640 1 -- 172.21.15.69:0/338726576 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fd00cad90 con 0x7f3fd00b2e60 2024-04-25T10:59:52.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.516+0000 7f3fd5d32640 1 --2- 172.21.15.69:0/338726576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fd00b2e60 0x7f3fd014df60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc00e8a0 tx=0x7f3fcc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:52.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.516+0000 7f3fc67fc640 1 -- 172.21.15.69:0/338726576 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc017070 con 0x7f3fd00b2e60 2024-04-25T10:59:52.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.516+0000 7f3fc67fc640 1 -- 172.21.15.69:0/338726576 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fcc00c940 con 0x7f3fd00b2e60 2024-04-25T10:59:52.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.516+0000 7f3fc67fc640 1 -- 172.21.15.69:0/338726576 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc00cd30 con 0x7f3fd00b2e60 2024-04-25T10:59:52.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.516+0000 7f3fd7f96640 1 -- 172.21.15.69:0/338726576 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fd00cb070 con 0x7f3fd00b2e60 2024-04-25T10:59:52.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.517+0000 7f3fd7f96640 1 -- 172.21.15.69:0/338726576 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fd00cb630 con 0x7f3fd00b2e60 2024-04-25T10:59:52.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.518+0000 7f3fd7f96640 1 -- 172.21.15.69:0/338726576 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fd0050740 con 0x7f3fd00b2e60 2024-04-25T10:59:52.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.518+0000 7f3fc67fc640 1 -- 172.21.15.69:0/338726576 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3fcc01a070 con 0x7f3fd00b2e60 2024-04-25T10:59:52.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.518+0000 7f3fc67fc640 1 --2- 172.21.15.69:0/338726576 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3fb40744e0 0x7f3fb40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:52.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.519+0000 7f3fc67fc640 1 -- 172.21.15.69:0/338726576 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3fcc095060 con 0x7f3fd00b2e60 2024-04-25T10:59:52.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.519+0000 7f3fd5531640 1 --2- 172.21.15.69:0/338726576 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3fb40744e0 0x7f3fb40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:52.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.519+0000 7f3fd5531640 1 --2- 172.21.15.69:0/338726576 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3fb40744e0 0x7f3fb40769f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0034000 tx=0x7f3fc002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:52.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.523+0000 7f3fc67fc640 1 -- 172.21.15.69:0/338726576 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3fcc05f880 con 0x7f3fd00b2e60 2024-04-25T10:59:52.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.699+0000 7f3fd7f96640 1 -- 172.21.15.69:0/338726576 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3fd0071000 con 0x7f3fd00b2e60 2024-04-25T10:59:52.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.700+0000 7f3fc67fc640 1 -- 172.21.15.69:0/338726576 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f3fcc05f180 con 0x7f3fd00b2e60 2024-04-25T10:59:52.700 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-25T10:59:52.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.702+0000 7f3fd7f96640 1 -- 172.21.15.69:0/338726576 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3fb40744e0 msgr2=0x7f3fb40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:52.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.702+0000 7f3fd7f96640 1 --2- 172.21.15.69:0/338726576 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3fb40744e0 0x7f3fb40769f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0034000 tx=0x7f3fc002e000 comp rx=0 tx=0).stop 2024-04-25T10:59:52.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.702+0000 7f3fd7f96640 1 -- 172.21.15.69:0/338726576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fd00b2e60 msgr2=0x7f3fd014df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:52.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.702+0000 7f3fd7f96640 1 --2- 172.21.15.69:0/338726576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fd00b2e60 0x7f3fd014df60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc00e8a0 tx=0x7f3fcc00ed60 comp rx=0 tx=0).stop 2024-04-25T10:59:52.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.702+0000 7f3fd7f96640 1 -- 172.21.15.69:0/338726576 shutdown_connections 2024-04-25T10:59:52.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.702+0000 7f3fd7f96640 1 --2- 172.21.15.69:0/338726576 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3fb40744e0 0x7f3fb40769f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:52.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.702+0000 7f3fd7f96640 1 --2- 172.21.15.69:0/338726576 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fd0164020 0x7f3fd00ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:52.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.702+0000 7f3fd7f96640 1 --2- 172.21.15.69:0/338726576 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3fd00d1e20 0x7f3fd00cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:52.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.702+0000 7f3fd7f96640 1 --2- 172.21.15.69:0/338726576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fd00b2e60 0x7f3fd014df60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:52.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.702+0000 7f3fd7f96640 1 -- 172.21.15.69:0/338726576 >> 172.21.15.69:0/338726576 conn(0x7f3fd00d0530 msgr2=0x7f3fd015ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:52.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.702+0000 7f3fd7f96640 1 -- 172.21.15.69:0/338726576 shutdown_connections 2024-04-25T10:59:52.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.702+0000 7f3fd7f96640 1 -- 172.21.15.69:0/338726576 wait complete. 2024-04-25T10:59:52.714 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:59:52.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.823+0000 7f22117d5640 1 Processor -- start 2024-04-25T10:59:52.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.823+0000 7f22117d5640 1 -- start start 2024-04-25T10:59:52.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.825+0000 7f22117d5640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f220c06dce0 0x7f220c0cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:52.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.825+0000 7f22117d5640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f220c15adf0 0x7f220c160d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:52.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.825+0000 7f22117d5640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f220c071110 con 0x7f220c070630 2024-04-25T10:59:52.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.825+0000 7f22117d5640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f220c0713f0 con 0x7f220c15adf0 2024-04-25T10:59:52.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.825+0000 7f22117d5640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f220c0701a0 con 0x7f220c06dce0 2024-04-25T10:59:52.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.825+0000 7f220b7fe640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f220c15adf0 0x7f220c160d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:52.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.825+0000 7f220b7fe640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f220c15adf0 0x7f220c160d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55240/0 (socket says 172.21.15.69:55240) 2024-04-25T10:59:52.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.825+0000 7f220b7fe640 1 -- 172.21.15.69:0/900335054 learned_addr learned my addr 172.21.15.69:0/900335054 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:52.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.825+0000 7f220a7fc640 1 --2- 172.21.15.69:0/900335054 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f220c06dce0 0x7f220c0cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:52.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.825+0000 7f220b7fe640 1 -- 172.21.15.69:0/900335054 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f220c06dce0 msgr2=0x7f220c0cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:52.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.825+0000 7f220b7fe640 1 --2- 172.21.15.69:0/900335054 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f220c06dce0 0x7f220c0cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:52.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.825+0000 7f220b7fe640 1 -- 172.21.15.69:0/900335054 >> v1:172.21.15.146:6789/0 conn(0x7f220c070630 legacy=0x7f220c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:52.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.825+0000 7f220b7fe640 1 -- 172.21.15.69:0/900335054 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f220c06fb50 con 0x7f220c15adf0 2024-04-25T10:59:52.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.826+0000 7f220b7fe640 1 --2- 172.21.15.69:0/900335054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f220c15adf0 0x7f220c160d80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2204009870 tx=0x7f220402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92bfcf439283d615 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:52.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.826+0000 7f2209ffb640 1 -- 172.21.15.69:0/900335054 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204037070 con 0x7f220c15adf0 2024-04-25T10:59:52.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.826+0000 7f2209ffb640 1 -- 172.21.15.69:0/900335054 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2204032440 con 0x7f220c15adf0 2024-04-25T10:59:52.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.826+0000 7f2209ffb640 1 -- 172.21.15.69:0/900335054 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204032810 con 0x7f220c15adf0 2024-04-25T10:59:52.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.827+0000 7f22117d5640 1 -- 172.21.15.69:0/900335054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f220c15adf0 msgr2=0x7f220c160d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:52.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.827+0000 7f22117d5640 1 --2- 172.21.15.69:0/900335054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f220c15adf0 0x7f220c160d80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2204009870 tx=0x7f220402a180 comp rx=0 tx=0).stop 2024-04-25T10:59:52.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.827+0000 7f22117d5640 1 -- 172.21.15.69:0/900335054 shutdown_connections 2024-04-25T10:59:52.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.827+0000 7f22117d5640 1 --2- 172.21.15.69:0/900335054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f220c15adf0 0x7f220c160d80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:52.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.827+0000 7f22117d5640 1 --2- 172.21.15.69:0/900335054 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f220c06dce0 0x7f220c0cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:52.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.827+0000 7f22117d5640 1 -- 172.21.15.69:0/900335054 >> 172.21.15.69:0/900335054 conn(0x7f220c073090 msgr2=0x7f220c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:52.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.827+0000 7f22117d5640 1 -- 172.21.15.69:0/900335054 shutdown_connections 2024-04-25T10:59:52.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.827+0000 7f22117d5640 1 -- 172.21.15.69:0/900335054 wait complete. 2024-04-25T10:59:52.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.829+0000 7f22117d5640 1 Processor -- start 2024-04-25T10:59:52.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.829+0000 7f22117d5640 1 -- start start 2024-04-25T10:59:52.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.830+0000 7f22117d5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f220c070630 0x7f220c14b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:52.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.830+0000 7f22117d5640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f220c15adf0 0x7f220c14cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:52.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.830+0000 7f22117d5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f220c0b48b0 0x7f220c0b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:52.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.830+0000 7f22117d5640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f220c0b0e80 con 0x7f220c0b48b0 2024-04-25T10:59:52.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.830+0000 7f22117d5640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f220c0b0d00 con 0x7f220c070630 2024-04-25T10:59:52.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.830+0000 7f22117d5640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f220c070c80 con 0x7f220c15adf0 2024-04-25T10:59:52.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.830+0000 7f220b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f220c0b48b0 0x7f220c0b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:52.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.830+0000 7f220b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f220c0b48b0 0x7f220c0b4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55244/0 (socket says 172.21.15.69:55244) 2024-04-25T10:59:52.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.830+0000 7f220b7fe640 1 -- 172.21.15.69:0/4257768870 learned_addr learned my addr 172.21.15.69:0/4257768870 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:52.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.830+0000 7f220a7fc640 1 --2- 172.21.15.69:0/4257768870 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f220c15adf0 0x7f220c14cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:52.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.830+0000 7f220b7fe640 1 -- 172.21.15.69:0/4257768870 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f220c15adf0 msgr2=0x7f220c14cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:52.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.830+0000 7f220affd640 1 --2- 172.21.15.69:0/4257768870 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f220c070630 0x7f220c14b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:52.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.830+0000 7f220b7fe640 1 --2- 172.21.15.69:0/4257768870 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f220c15adf0 0x7f220c14cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:52.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.830+0000 7f220b7fe640 1 -- 172.21.15.69:0/4257768870 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f220c070630 msgr2=0x7f220c14b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:52.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.830+0000 7f220b7fe640 1 --2- 172.21.15.69:0/4257768870 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f220c070630 0x7f220c14b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:52.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.830+0000 7f220b7fe640 1 -- 172.21.15.69:0/4257768870 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f220c14d000 con 0x7f220c0b48b0 2024-04-25T10:59:52.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.830+0000 7f220a7fc640 1 --2- 172.21.15.69:0/4257768870 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f220c15adf0 0x7f220c14cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:52.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.830+0000 7f220affd640 1 --2- 172.21.15.69:0/4257768870 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f220c070630 0x7f220c14b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:52.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.831+0000 7f220b7fe640 1 --2- 172.21.15.69:0/4257768870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f220c0b48b0 0x7f220c0b4cc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f220402a1c0 tx=0x7f22040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:52.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.831+0000 7f21f37fe640 1 -- 172.21.15.69:0/4257768870 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204041020 con 0x7f220c0b48b0 2024-04-25T10:59:52.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.831+0000 7f21f37fe640 1 -- 172.21.15.69:0/4257768870 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22040306c0 con 0x7f220c0b48b0 2024-04-25T10:59:52.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.831+0000 7f22117d5640 1 -- 172.21.15.69:0/4257768870 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f220c0b5200 con 0x7f220c0b48b0 2024-04-25T10:59:52.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.831+0000 7f21f37fe640 1 -- 172.21.15.69:0/4257768870 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204037040 con 0x7f220c0b48b0 2024-04-25T10:59:52.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.831+0000 7f22117d5640 1 -- 172.21.15.69:0/4257768870 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f220c1f4900 con 0x7f220c0b48b0 2024-04-25T10:59:52.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.832+0000 7f22117d5640 1 -- 172.21.15.69:0/4257768870 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f220c1f4380 con 0x7f220c0b48b0 2024-04-25T10:59:52.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.832+0000 7f21f37fe640 1 -- 172.21.15.69:0/4257768870 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2204044020 con 0x7f220c0b48b0 2024-04-25T10:59:52.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.833+0000 7f21f37fe640 1 --2- 172.21.15.69:0/4257768870 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f21f80744e0 0x7f21f80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:52.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.833+0000 7f21f37fe640 1 -- 172.21.15.69:0/4257768870 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f22040b5970 con 0x7f220c0b48b0 2024-04-25T10:59:52.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.833+0000 7f220affd640 1 --2- 172.21.15.69:0/4257768870 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f21f80744e0 0x7f21f80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:52.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.833+0000 7f220affd640 1 --2- 172.21.15.69:0/4257768870 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f21f80744e0 0x7f21f80769f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f220c0cfd00 tx=0x7f21fc0073a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:52.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:52.837+0000 7f21f37fe640 1 -- 172.21.15.69:0/4257768870 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f220c1f4380 con 0x7f220c0b48b0 2024-04-25T10:59:53.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.010+0000 7f22117d5640 1 -- 172.21.15.69:0/4257768870 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f220c050740 con 0x7f220c0b48b0 2024-04-25T10:59:53.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.011+0000 7f21f37fe640 1 -- 172.21.15.69:0/4257768870 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f220404b030 con 0x7f220c0b48b0 2024-04-25T10:59:53.012 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-25T10:59:53.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.013+0000 7f22117d5640 1 -- 172.21.15.69:0/4257768870 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f21f80744e0 msgr2=0x7f21f80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:53.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.013+0000 7f22117d5640 1 --2- 172.21.15.69:0/4257768870 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f21f80744e0 0x7f21f80769f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f220c0cfd00 tx=0x7f21fc0073a0 comp rx=0 tx=0).stop 2024-04-25T10:59:53.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.013+0000 7f22117d5640 1 -- 172.21.15.69:0/4257768870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f220c0b48b0 msgr2=0x7f220c0b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:53.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.013+0000 7f22117d5640 1 --2- 172.21.15.69:0/4257768870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f220c0b48b0 0x7f220c0b4cc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f220402a1c0 tx=0x7f22040046c0 comp rx=0 tx=0).stop 2024-04-25T10:59:53.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.013+0000 7f22117d5640 1 -- 172.21.15.69:0/4257768870 shutdown_connections 2024-04-25T10:59:53.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.013+0000 7f22117d5640 1 --2- 172.21.15.69:0/4257768870 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f21f80744e0 0x7f21f80769f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:53.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.013+0000 7f22117d5640 1 --2- 172.21.15.69:0/4257768870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f220c0b48b0 0x7f220c0b4cc0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:53.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.013+0000 7f22117d5640 1 --2- 172.21.15.69:0/4257768870 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f220c15adf0 0x7f220c14cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:53.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.013+0000 7f22117d5640 1 --2- 172.21.15.69:0/4257768870 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f220c070630 0x7f220c14b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:53.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.013+0000 7f22117d5640 1 -- 172.21.15.69:0/4257768870 >> 172.21.15.69:0/4257768870 conn(0x7f220c073090 msgr2=0x7f220c06d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:53.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.013+0000 7f22117d5640 1 -- 172.21.15.69:0/4257768870 shutdown_connections 2024-04-25T10:59:53.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.013+0000 7f22117d5640 1 -- 172.21.15.69:0/4257768870 wait complete. 2024-04-25T10:59:53.025 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:59:53.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.132+0000 7fe34a4bf640 1 Processor -- start 2024-04-25T10:59:53.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.133+0000 7fe34a4bf640 1 -- start start 2024-04-25T10:59:53.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.134+0000 7fe34a4bf640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe344164b00 0x7fe3440b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:53.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.134+0000 7fe34a4bf640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3441615f0 con 0x7fe344160fa0 2024-04-25T10:59:53.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.134+0000 7fe34a4bf640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe344164020 con 0x7fe34415df80 2024-04-25T10:59:53.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.134+0000 7fe34a4bf640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fe3441613b0 con 0x7fe344164b00 2024-04-25T10:59:53.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.134+0000 7fe343fff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe344164b00 0x7fe3440b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:53.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.134+0000 7fe343fff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fe344164b00 0x7fe3440b7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55252/0 (socket says 172.21.15.69:55252) 2024-04-25T10:59:53.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.134+0000 7fe343fff640 1 -- 172.21.15.69:0/3887865 learned_addr learned my addr 172.21.15.69:0/3887865 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:53.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.134+0000 7fe348a5c640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fe344160fa0 0x7fe3440b4880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:50214/0 (socket says 172.21.15.69:50214) 2024-04-25T10:59:53.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.134+0000 7fe343fff640 1 -- 172.21.15.69:0/3887865 >> v1:172.21.15.146:6789/0 conn(0x7fe34415df80 legacy=0x7fe344160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:53.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.134+0000 7fe343fff640 1 -- 172.21.15.69:0/3887865 >> v1:172.21.15.69:6789/0 conn(0x7fe344160fa0 legacy=0x7fe3440b4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:53.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.134+0000 7fe343fff640 1 -- 172.21.15.69:0/3887865 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe344164300 con 0x7fe344164b00 2024-04-25T10:59:53.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.134+0000 7fe342ffd640 1 -- 172.21.15.69:0/3887865 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1826150339 0 0) 0x7fe3441615f0 con 0x7fe344160fa0 2024-04-25T10:59:53.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.135+0000 7fe343fff640 1 --2- 172.21.15.69:0/3887865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe344164b00 0x7fe3440b7b80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe334009870 tx=0x7fe33402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=449cc0960bddba56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:53.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.135+0000 7fe342ffd640 1 -- 172.21.15.69:0/3887865 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe334037070 con 0x7fe344164b00 2024-04-25T10:59:53.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.135+0000 7fe342ffd640 1 -- 172.21.15.69:0/3887865 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe334032440 con 0x7fe344164b00 2024-04-25T10:59:53.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.135+0000 7fe342ffd640 1 -- 172.21.15.69:0/3887865 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe334032810 con 0x7fe344164b00 2024-04-25T10:59:53.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.136+0000 7fe34a4bf640 1 -- 172.21.15.69:0/3887865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe344164b00 msgr2=0x7fe3440b7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:53.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.136+0000 7fe34a4bf640 1 --2- 172.21.15.69:0/3887865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe344164b00 0x7fe3440b7b80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe334009870 tx=0x7fe33402a2a0 comp rx=0 tx=0).stop 2024-04-25T10:59:53.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.136+0000 7fe34a4bf640 1 -- 172.21.15.69:0/3887865 shutdown_connections 2024-04-25T10:59:53.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.136+0000 7fe34a4bf640 1 --2- 172.21.15.69:0/3887865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe344164b00 0x7fe3440b7b80 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:53.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.136+0000 7fe34a4bf640 1 -- 172.21.15.69:0/3887865 >> 172.21.15.69:0/3887865 conn(0x7fe3440d2bc0 msgr2=0x7fe3440d2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:53.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.136+0000 7fe34a4bf640 1 -- 172.21.15.69:0/3887865 shutdown_connections 2024-04-25T10:59:53.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.136+0000 7fe34a4bf640 1 -- 172.21.15.69:0/3887865 wait complete. 2024-04-25T10:59:53.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.138+0000 7fe34a4bf640 1 Processor -- start 2024-04-25T10:59:53.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.138+0000 7fe34a4bf640 1 -- start start 2024-04-25T10:59:53.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.139+0000 7fe34a4bf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe34415df80 0x7fe3441f9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:53.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.139+0000 7fe34a4bf640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe344160fa0 0x7fe3440adf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:53.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.139+0000 7fe34a4bf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe344164b00 0x7fe3441616f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:53.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.139+0000 7fe34a4bf640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe3440b84d0 con 0x7fe34415df80 2024-04-25T10:59:53.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.139+0000 7fe34a4bf640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe3440b8350 con 0x7fe344164b00 2024-04-25T10:59:53.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.139+0000 7fe34a4bf640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe344164670 con 0x7fe344160fa0 2024-04-25T10:59:53.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.139+0000 7fe343fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe34415df80 0x7fe3441f9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:53.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.139+0000 7fe343fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe34415df80 0x7fe3441f9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55264/0 (socket says 172.21.15.69:55264) 2024-04-25T10:59:53.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.139+0000 7fe343fff640 1 -- 172.21.15.69:0/4144477164 learned_addr learned my addr 172.21.15.69:0/4144477164 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:53.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.139+0000 7fe348a5c640 1 --2- 172.21.15.69:0/4144477164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe344164b00 0x7fe3441616f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:53.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.139+0000 7fe3437fe640 1 --2- 172.21.15.69:0/4144477164 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe344160fa0 0x7fe3440adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:53.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.139+0000 7fe343fff640 1 -- 172.21.15.69:0/4144477164 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe344160fa0 msgr2=0x7fe3440adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:53.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.139+0000 7fe343fff640 1 --2- 172.21.15.69:0/4144477164 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe344160fa0 0x7fe3440adf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:53.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.139+0000 7fe343fff640 1 -- 172.21.15.69:0/4144477164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe344164b00 msgr2=0x7fe3441616f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:53.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.139+0000 7fe343fff640 1 --2- 172.21.15.69:0/4144477164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe344164b00 0x7fe3441616f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:53.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.139+0000 7fe343fff640 1 -- 172.21.15.69:0/4144477164 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3441f9680 con 0x7fe34415df80 2024-04-25T10:59:53.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.140+0000 7fe343fff640 1 --2- 172.21.15.69:0/4144477164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe34415df80 0x7fe3441f9140 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe334009870 tx=0x7fe3340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:53.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.140+0000 7fe340ff9640 1 -- 172.21.15.69:0/4144477164 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe334037040 con 0x7fe34415df80 2024-04-25T10:59:53.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.140+0000 7fe340ff9640 1 -- 172.21.15.69:0/4144477164 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe334004250 con 0x7fe34415df80 2024-04-25T10:59:53.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.140+0000 7fe34a4bf640 1 -- 172.21.15.69:0/4144477164 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3440acce0 con 0x7fe34415df80 2024-04-25T10:59:53.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.140+0000 7fe340ff9640 1 -- 172.21.15.69:0/4144477164 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33403c670 con 0x7fe34415df80 2024-04-25T10:59:53.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.140+0000 7fe34a4bf640 1 -- 172.21.15.69:0/4144477164 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3440ad2a0 con 0x7fe34415df80 2024-04-25T10:59:53.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.141+0000 7fe34a4bf640 1 -- 172.21.15.69:0/4144477164 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe344050740 con 0x7fe34415df80 2024-04-25T10:59:53.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.142+0000 7fe340ff9640 1 -- 172.21.15.69:0/4144477164 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe33403a070 con 0x7fe34415df80 2024-04-25T10:59:53.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.142+0000 7fe340ff9640 1 --2- 172.21.15.69:0/4144477164 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe3300744b0 0x7fe3300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:53.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.142+0000 7fe340ff9640 1 -- 172.21.15.69:0/4144477164 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe3340b5b80 con 0x7fe34415df80 2024-04-25T10:59:53.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.142+0000 7fe3437fe640 1 --2- 172.21.15.69:0/4144477164 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe3300744b0 0x7fe3300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:53.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.143+0000 7fe3437fe640 1 --2- 172.21.15.69:0/4144477164 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe3300744b0 0x7fe3300769c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe338006fd0 tx=0x7fe338008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:53.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.147+0000 7fe340ff9640 1 -- 172.21.15.69:0/4144477164 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe334041020 con 0x7fe34415df80 2024-04-25T10:59:53.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.333+0000 7fe34a4bf640 1 -- 172.21.15.69:0/4144477164 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe3440c8fa0 con 0x7fe34415df80 2024-04-25T10:59:53.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.333+0000 7fe340ff9640 1 -- 172.21.15.69:0/4144477164 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7fe334080310 con 0x7fe34415df80 2024-04-25T10:59:53.334 INFO:teuthology.orchestra.run.smithi069.stderr:listed 0 entries 2024-04-25T10:59:53.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.335+0000 7fe34a4bf640 1 -- 172.21.15.69:0/4144477164 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe3300744b0 msgr2=0x7fe3300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:53.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.335+0000 7fe34a4bf640 1 --2- 172.21.15.69:0/4144477164 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe3300744b0 0x7fe3300769c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe338006fd0 tx=0x7fe338008040 comp rx=0 tx=0).stop 2024-04-25T10:59:53.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.335+0000 7fe34a4bf640 1 -- 172.21.15.69:0/4144477164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe34415df80 msgr2=0x7fe3441f9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:53.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.335+0000 7fe34a4bf640 1 --2- 172.21.15.69:0/4144477164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe34415df80 0x7fe3441f9140 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe334009870 tx=0x7fe3340046c0 comp rx=0 tx=0).stop 2024-04-25T10:59:53.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.335+0000 7fe34a4bf640 1 -- 172.21.15.69:0/4144477164 shutdown_connections 2024-04-25T10:59:53.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.335+0000 7fe34a4bf640 1 --2- 172.21.15.69:0/4144477164 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe3300744b0 0x7fe3300769c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:53.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.335+0000 7fe34a4bf640 1 --2- 172.21.15.69:0/4144477164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe344164b00 0x7fe3441616f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:53.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.335+0000 7fe34a4bf640 1 --2- 172.21.15.69:0/4144477164 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe344160fa0 0x7fe3440adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:53.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.335+0000 7fe34a4bf640 1 --2- 172.21.15.69:0/4144477164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe34415df80 0x7fe3441f9140 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:53.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.335+0000 7fe34a4bf640 1 -- 172.21.15.69:0/4144477164 >> 172.21.15.69:0/4144477164 conn(0x7fe3440d2bc0 msgr2=0x7fe3440d4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:53.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.335+0000 7fe34a4bf640 1 -- 172.21.15.69:0/4144477164 shutdown_connections 2024-04-25T10:59:53.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.335+0000 7fe34a4bf640 1 -- 172.21.15.69:0/4144477164 wait complete. 2024-04-25T10:59:53.347 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-25T10:59:53.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:53.347 DEBUG:teuthology.orchestra.run.smithi146:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-25T10:59:53.378 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2024-04-25T10:59:53.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:59:53.378 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:59:53.436 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:59:53.436 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:59:53.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.542+0000 7fdf80db7640 1 Processor -- start 2024-04-25T10:59:53.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.542+0000 7fdf80db7640 1 -- start start 2024-04-25T10:59:53.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.543+0000 7fdf80db7640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdf7c0b6a00 0x7fdf7c15d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:53.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.543+0000 7fdf80db7640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fdf7c06d560 0x7fdf7c0b4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:53.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.543+0000 7fdf80db7640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf7c1611e0 con 0x7fdf7c15a2f0 2024-04-25T10:59:53.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.543+0000 7fdf80db7640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fdf7c160d50 con 0x7fdf7c06d560 2024-04-25T10:59:53.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.543+0000 7fdf80db7640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fdf7c15a700 con 0x7fdf7c0b6a00 2024-04-25T10:59:53.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.543+0000 7fdf7ad9d640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fdf7c06d560 0x7fdf7c0b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:53.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.543+0000 7fdf7ad9d640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fdf7c06d560 0x7fdf7c0b4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55280/0 (socket says 172.21.15.69:55280) 2024-04-25T10:59:53.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.543+0000 7fdf7ad9d640 1 -- 172.21.15.69:0/394046226 learned_addr learned my addr 172.21.15.69:0/394046226 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:53.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.543+0000 7fdf73fff640 1 --1- 172.21.15.69:0/394046226 >> v1:172.21.15.69:6789/0 conn(0x7fdf7c15a2f0 0x7fdf7c06ce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:50220/0 (socket says 172.21.15.69:50220) 2024-04-25T10:59:53.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.544+0000 7fdf7ad9d640 1 -- 172.21.15.69:0/394046226 >> v2:172.21.15.146:3300/0 conn(0x7fdf7c0b6a00 msgr2=0x7fdf7c15d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:53.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.544+0000 7fdf7ad9d640 1 --2- 172.21.15.69:0/394046226 >> v2:172.21.15.146:3300/0 conn(0x7fdf7c0b6a00 0x7fdf7c15d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:53.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.544+0000 7fdf7ad9d640 1 -- 172.21.15.69:0/394046226 >> v1:172.21.15.69:6789/0 conn(0x7fdf7c15a2f0 legacy=0x7fdf7c06ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:53.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.544+0000 7fdf79d9b640 1 -- 172.21.15.69:0/394046226 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1671403419 0 0) 0x7fdf7c1611e0 con 0x7fdf7c15a2f0 2024-04-25T10:59:53.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.544+0000 7fdf7ad9d640 1 -- 172.21.15.69:0/394046226 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7c160420 con 0x7fdf7c06d560 2024-04-25T10:59:53.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.544+0000 7fdf7ad9d640 1 --2- 172.21.15.69:0/394046226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf7c06d560 0x7fdf7c0b4970 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0713e0 tx=0x7fdf7402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d1ff6cda6bb53d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:53.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.545+0000 7fdf79d9b640 1 -- 172.21.15.69:0/394046226 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf74036070 con 0x7fdf7c06d560 2024-04-25T10:59:53.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.545+0000 7fdf79d9b640 1 -- 172.21.15.69:0/394046226 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf74031600 con 0x7fdf7c06d560 2024-04-25T10:59:53.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.545+0000 7fdf79d9b640 1 -- 172.21.15.69:0/394046226 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf740319d0 con 0x7fdf7c06d560 2024-04-25T10:59:53.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.545+0000 7fdf80db7640 1 -- 172.21.15.69:0/394046226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf7c06d560 msgr2=0x7fdf7c0b4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:53.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.545+0000 7fdf80db7640 1 --2- 172.21.15.69:0/394046226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf7c06d560 0x7fdf7c0b4970 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0713e0 tx=0x7fdf7402a730 comp rx=0 tx=0).stop 2024-04-25T10:59:53.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.545+0000 7fdf80db7640 1 -- 172.21.15.69:0/394046226 shutdown_connections 2024-04-25T10:59:53.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.545+0000 7fdf80db7640 1 --2- 172.21.15.69:0/394046226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf7c06d560 0x7fdf7c0b4970 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:53.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.545+0000 7fdf80db7640 1 --2- 172.21.15.69:0/394046226 >> v2:172.21.15.146:3300/0 conn(0x7fdf7c0b6a00 0x7fdf7c15d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:53.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.545+0000 7fdf80db7640 1 -- 172.21.15.69:0/394046226 >> 172.21.15.69:0/394046226 conn(0x7fdf7c0d1ae0 msgr2=0x7fdf7c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:53.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.545+0000 7fdf80db7640 1 -- 172.21.15.69:0/394046226 shutdown_connections 2024-04-25T10:59:53.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.545+0000 7fdf80db7640 1 -- 172.21.15.69:0/394046226 wait complete. 2024-04-25T10:59:53.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.547+0000 7fdf80db7640 1 Processor -- start 2024-04-25T10:59:53.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.547+0000 7fdf80db7640 1 -- start start 2024-04-25T10:59:53.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.549+0000 7fdf80db7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf7c06d560 0x7fdf7c14d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:53.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.549+0000 7fdf80db7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf7c0b6a00 0x7fdf7c0732b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:53.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.549+0000 7fdf80db7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf7c15a2f0 0x7fdf7c0737f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:53.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.549+0000 7fdf80db7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7c0ce580 con 0x7fdf7c15a2f0 2024-04-25T10:59:53.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.549+0000 7fdf80db7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7c1609e0 con 0x7fdf7c06d560 2024-04-25T10:59:53.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.549+0000 7fdf80db7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fdf7c0ce700 con 0x7fdf7c0b6a00 2024-04-25T10:59:53.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.549+0000 7fdf7ad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf7c15a2f0 0x7fdf7c0737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:53.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.549+0000 7fdf7ad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf7c15a2f0 0x7fdf7c0737f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55286/0 (socket says 172.21.15.69:55286) 2024-04-25T10:59:53.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.549+0000 7fdf7ad9d640 1 -- 172.21.15.69:0/2100222628 learned_addr learned my addr 172.21.15.69:0/2100222628 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:53.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.549+0000 7fdf73fff640 1 --2- 172.21.15.69:0/2100222628 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf7c0b6a00 0x7fdf7c0732b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:53.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.549+0000 7fdf7a59c640 1 --2- 172.21.15.69:0/2100222628 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf7c06d560 0x7fdf7c14d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:53.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.549+0000 7fdf7ad9d640 1 -- 172.21.15.69:0/2100222628 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf7c0b6a00 msgr2=0x7fdf7c0732b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:53.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.549+0000 7fdf7ad9d640 1 --2- 172.21.15.69:0/2100222628 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf7c0b6a00 0x7fdf7c0732b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:53.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.549+0000 7fdf7ad9d640 1 -- 172.21.15.69:0/2100222628 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf7c06d560 msgr2=0x7fdf7c14d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:53.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.549+0000 7fdf7ad9d640 1 --2- 172.21.15.69:0/2100222628 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf7c06d560 0x7fdf7c14d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:53.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.549+0000 7fdf7ad9d640 1 -- 172.21.15.69:0/2100222628 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7c15ac20 con 0x7fdf7c15a2f0 2024-04-25T10:59:53.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.549+0000 7fdf7a59c640 1 --2- 172.21.15.69:0/2100222628 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf7c06d560 0x7fdf7c14d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:53.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.549+0000 7fdf73fff640 1 --2- 172.21.15.69:0/2100222628 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf7c0b6a00 0x7fdf7c0732b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:53.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.549+0000 7fdf7ad9d640 1 --2- 172.21.15.69:0/2100222628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf7c15a2f0 0x7fdf7c0737f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdf74009f40 tx=0x7fdf74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:53.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.549+0000 7fdf72ffd640 1 -- 172.21.15.69:0/2100222628 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf74041070 con 0x7fdf7c15a2f0 2024-04-25T10:59:53.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.549+0000 7fdf72ffd640 1 -- 172.21.15.69:0/2100222628 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf74003990 con 0x7fdf7c15a2f0 2024-04-25T10:59:53.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.549+0000 7fdf80db7640 1 -- 172.21.15.69:0/2100222628 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf7c15af00 con 0x7fdf7c15a2f0 2024-04-25T10:59:53.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.549+0000 7fdf72ffd640 1 -- 172.21.15.69:0/2100222628 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf74036040 con 0x7fdf7c15a2f0 2024-04-25T10:59:53.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.550+0000 7fdf80db7640 1 -- 172.21.15.69:0/2100222628 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf7c15b4c0 con 0x7fdf7c15a2f0 2024-04-25T10:59:53.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.550+0000 7fdf80db7640 1 -- 172.21.15.69:0/2100222628 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf7c15a990 con 0x7fdf7c15a2f0 2024-04-25T10:59:53.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.551+0000 7fdf72ffd640 1 -- 172.21.15.69:0/2100222628 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdf74007820 con 0x7fdf7c15a2f0 2024-04-25T10:59:53.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.552+0000 7fdf72ffd640 1 --2- 172.21.15.69:0/2100222628 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fdf5c0744b0 0x7fdf5c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:53.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.552+0000 7fdf72ffd640 1 -- 172.21.15.69:0/2100222628 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdf740b5e60 con 0x7fdf7c15a2f0 2024-04-25T10:59:53.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.552+0000 7fdf7a59c640 1 --2- 172.21.15.69:0/2100222628 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fdf5c0744b0 0x7fdf5c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:53.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.552+0000 7fdf7a59c640 1 --2- 172.21.15.69:0/2100222628 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fdf5c0744b0 0x7fdf5c0769c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c070570 tx=0x7fdf6c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:53.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.556+0000 7fdf72ffd640 1 -- 172.21.15.69:0/2100222628 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdf7404b020 con 0x7fdf7c15a2f0 2024-04-25T10:59:53.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.772+0000 7fdf80db7640 1 -- 172.21.15.69:0/2100222628 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf7c0705a0 con 0x7fdf7c15a2f0 2024-04-25T10:59:53.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.773+0000 7fdf72ffd640 1 -- 172.21.15.69:0/2100222628 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5178 (secure 0 0 0) 0x7fdf740805f0 con 0x7fdf7c15a2f0 2024-04-25T10:59:53.774 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:53.774 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/2361834023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2361834023},{"type":"v1","addr":"172.21.15.146:6833","nonce":2361834023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1848276963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1848276963},{"type":"v1","addr":"172.21.15.69:6837","nonce":1848276963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1826742846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1826742846},{"type":"v1","addr":"172.21.15.69:6836","nonce":1826742846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:59:18.976232+0000","modified":"2024-04-25T10:59:20.983284+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6835/908008797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":908008797},{"type":"v1","addr":"172.21.15.146:6835","nonce":908008797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-04-25T10:59:53.774 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 6 2024-04-25T10:59:53.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.775+0000 7fdf80db7640 1 -- 172.21.15.69:0/2100222628 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fdf5c0744b0 msgr2=0x7fdf5c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:53.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.775+0000 7fdf80db7640 1 --2- 172.21.15.69:0/2100222628 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fdf5c0744b0 0x7fdf5c0769c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c070570 tx=0x7fdf6c008040 comp rx=0 tx=0).stop 2024-04-25T10:59:53.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.775+0000 7fdf80db7640 1 -- 172.21.15.69:0/2100222628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf7c15a2f0 msgr2=0x7fdf7c0737f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:53.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.775+0000 7fdf80db7640 1 --2- 172.21.15.69:0/2100222628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf7c15a2f0 0x7fdf7c0737f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdf74009f40 tx=0x7fdf74004a40 comp rx=0 tx=0).stop 2024-04-25T10:59:53.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.775+0000 7fdf80db7640 1 -- 172.21.15.69:0/2100222628 shutdown_connections 2024-04-25T10:59:53.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.775+0000 7fdf80db7640 1 --2- 172.21.15.69:0/2100222628 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fdf5c0744b0 0x7fdf5c0769c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:53.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.775+0000 7fdf80db7640 1 --2- 172.21.15.69:0/2100222628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf7c15a2f0 0x7fdf7c0737f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:53.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.775+0000 7fdf80db7640 1 --2- 172.21.15.69:0/2100222628 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf7c0b6a00 0x7fdf7c0732b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:53.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.775+0000 7fdf80db7640 1 --2- 172.21.15.69:0/2100222628 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf7c06d560 0x7fdf7c14d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:53.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.775+0000 7fdf80db7640 1 -- 172.21.15.69:0/2100222628 >> 172.21.15.69:0/2100222628 conn(0x7fdf7c0d1ae0 msgr2=0x7fdf7c0d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:53.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.775+0000 7fdf80db7640 1 -- 172.21.15.69:0/2100222628 shutdown_connections 2024-04-25T10:59:53.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.775+0000 7fdf80db7640 1 -- 172.21.15.69:0/2100222628 wait complete. 2024-04-25T10:59:53.787 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:59:53.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.894+0000 7fb981451640 1 Processor -- start 2024-04-25T10:59:53.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.894+0000 7fb981451640 1 -- start start 2024-04-25T10:59:53.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.896+0000 7fb981451640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb97c0d0e00 0x7fb97c0d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:53.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.896+0000 7fb981451640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb97c0d4a50 0x7fb97c0d4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:53.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.896+0000 7fb981451640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb97c0d1850 con 0x7fb97c06c5c0 2024-04-25T10:59:53.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.896+0000 7fb981451640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fb97c160de0 con 0x7fb97c0d4a50 2024-04-25T10:59:53.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.896+0000 7fb981451640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fb97c1610c0 con 0x7fb97c0d0e00 2024-04-25T10:59:53.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.896+0000 7fb97b7fe640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb97c0d4a50 0x7fb97c0d4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:53.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.896+0000 7fb97a7fc640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fb97c06c5c0 0x7fb97c06c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:50230/0 (socket says 172.21.15.69:50230) 2024-04-25T10:59:53.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.896+0000 7fb97a7fc640 1 -- 172.21.15.69:0/715271110 learned_addr learned my addr 172.21.15.69:0/715271110 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:53.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.896+0000 7fb97affd640 1 --2- 172.21.15.69:0/715271110 >> v2:172.21.15.146:3300/0 conn(0x7fb97c0d0e00 0x7fb97c0d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:53.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.896+0000 7fb97b7fe640 1 -- 172.21.15.69:0/715271110 >> v2:172.21.15.146:3300/0 conn(0x7fb97c0d0e00 msgr2=0x7fb97c0d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:53.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.896+0000 7fb97b7fe640 1 --2- 172.21.15.69:0/715271110 >> v2:172.21.15.146:3300/0 conn(0x7fb97c0d0e00 0x7fb97c0d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:53.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.896+0000 7fb97b7fe640 1 -- 172.21.15.69:0/715271110 >> v1:172.21.15.69:6789/0 conn(0x7fb97c06c5c0 legacy=0x7fb97c06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:53.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.896+0000 7fb979ffb640 1 -- 172.21.15.69:0/715271110 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2522830734 0 0) 0x7fb97c0d1850 con 0x7fb97c06c5c0 2024-04-25T10:59:53.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.896+0000 7fb97b7fe640 1 -- 172.21.15.69:0/715271110 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97c1613a0 con 0x7fb97c0d4a50 2024-04-25T10:59:53.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.897+0000 7fb97b7fe640 1 --2- 172.21.15.69:0/715271110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb97c0d4a50 0x7fb97c0d4e60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb97c0d3e60 tx=0x7fb97402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30175157d589d85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:53.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.897+0000 7fb979ffb640 1 -- 172.21.15.69:0/715271110 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb974037070 con 0x7fb97c0d4a50 2024-04-25T10:59:53.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.897+0000 7fb979ffb640 1 -- 172.21.15.69:0/715271110 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb974032440 con 0x7fb97c0d4a50 2024-04-25T10:59:53.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.897+0000 7fb979ffb640 1 -- 172.21.15.69:0/715271110 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97402ae60 con 0x7fb97c0d4a50 2024-04-25T10:59:53.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.898+0000 7fb981451640 1 -- 172.21.15.69:0/715271110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb97c0d4a50 msgr2=0x7fb97c0d4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:53.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.898+0000 7fb981451640 1 --2- 172.21.15.69:0/715271110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb97c0d4a50 0x7fb97c0d4e60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb97c0d3e60 tx=0x7fb97402a110 comp rx=0 tx=0).stop 2024-04-25T10:59:53.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.898+0000 7fb981451640 1 -- 172.21.15.69:0/715271110 shutdown_connections 2024-04-25T10:59:53.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.898+0000 7fb981451640 1 --2- 172.21.15.69:0/715271110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb97c0d4a50 0x7fb97c0d4e60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:53.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.898+0000 7fb981451640 1 --2- 172.21.15.69:0/715271110 >> v2:172.21.15.146:3300/0 conn(0x7fb97c0d0e00 0x7fb97c0d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:53.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.898+0000 7fb981451640 1 -- 172.21.15.69:0/715271110 >> 172.21.15.69:0/715271110 conn(0x7fb97c0d5d90 msgr2=0x7fb97c0d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:53.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.898+0000 7fb981451640 1 -- 172.21.15.69:0/715271110 shutdown_connections 2024-04-25T10:59:53.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.898+0000 7fb981451640 1 -- 172.21.15.69:0/715271110 wait complete. 2024-04-25T10:59:53.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.900+0000 7fb981451640 1 Processor -- start 2024-04-25T10:59:53.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.900+0000 7fb981451640 1 -- start start 2024-04-25T10:59:53.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.901+0000 7fb981451640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb97c06c5c0 0x7fb97c0c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:53.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.901+0000 7fb981451640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb97c0d0e00 0x7fb97c1512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:53.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.901+0000 7fb981451640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb97c0d4a50 0x7fb97c0cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:53.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.901+0000 7fb981451640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c164e00 con 0x7fb97c0d0e00 2024-04-25T10:59:53.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.901+0000 7fb981451640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c0b5320 con 0x7fb97c0d4a50 2024-04-25T10:59:53.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.901+0000 7fb981451640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb97c164f80 con 0x7fb97c06c5c0 2024-04-25T10:59:53.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.901+0000 7fb97a7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb97c0d0e00 0x7fb97c1512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:53.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.901+0000 7fb97a7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb97c0d0e00 0x7fb97c1512e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55304/0 (socket says 172.21.15.69:55304) 2024-04-25T10:59:53.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.901+0000 7fb97a7fc640 1 -- 172.21.15.69:0/2136138758 learned_addr learned my addr 172.21.15.69:0/2136138758 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:53.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.901+0000 7fb97affd640 1 --2- 172.21.15.69:0/2136138758 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb97c06c5c0 0x7fb97c0c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:53.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.901+0000 7fb97b7fe640 1 --2- 172.21.15.69:0/2136138758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb97c0d4a50 0x7fb97c0cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:53.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.901+0000 7fb97a7fc640 1 -- 172.21.15.69:0/2136138758 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb97c06c5c0 msgr2=0x7fb97c0c94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:53.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.901+0000 7fb97a7fc640 1 --2- 172.21.15.69:0/2136138758 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb97c06c5c0 0x7fb97c0c94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:53.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.901+0000 7fb97a7fc640 1 -- 172.21.15.69:0/2136138758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb97c0d4a50 msgr2=0x7fb97c0cba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:53.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.901+0000 7fb97a7fc640 1 --2- 172.21.15.69:0/2136138758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb97c0d4a50 0x7fb97c0cba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:53.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.901+0000 7fb97a7fc640 1 -- 172.21.15.69:0/2136138758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97c0c99f0 con 0x7fb97c0d0e00 2024-04-25T10:59:53.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.901+0000 7fb97b7fe640 1 --2- 172.21.15.69:0/2136138758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb97c0d4a50 0x7fb97c0cba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:53.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.902+0000 7fb97a7fc640 1 --2- 172.21.15.69:0/2136138758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb97c0d0e00 0x7fb97c1512e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb97000c430 tx=0x7fb97000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:53.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.902+0000 7fb9637fe640 1 -- 172.21.15.69:0/2136138758 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9700074e0 con 0x7fb97c0d0e00 2024-04-25T10:59:53.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.902+0000 7fb9637fe640 1 -- 172.21.15.69:0/2136138758 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9700047e0 con 0x7fb97c0d0e00 2024-04-25T10:59:53.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.902+0000 7fb9637fe640 1 -- 172.21.15.69:0/2136138758 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb970004bd0 con 0x7fb97c0d0e00 2024-04-25T10:59:53.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.902+0000 7fb981451640 1 -- 172.21.15.69:0/2136138758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb97c0c9f00 con 0x7fb97c0d0e00 2024-04-25T10:59:53.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.902+0000 7fb981451640 1 -- 172.21.15.69:0/2136138758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb97c0ca4c0 con 0x7fb97c0d0e00 2024-04-25T10:59:53.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.903+0000 7fb981451640 1 -- 172.21.15.69:0/2136138758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb97c0c9c70 con 0x7fb97c0d0e00 2024-04-25T10:59:53.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.904+0000 7fb9637fe640 1 -- 172.21.15.69:0/2136138758 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb970011070 con 0x7fb97c0d0e00 2024-04-25T10:59:53.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.904+0000 7fb9637fe640 1 --2- 172.21.15.69:0/2136138758 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb9680744e0 0x7fb9680769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:53.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.904+0000 7fb9637fe640 1 -- 172.21.15.69:0/2136138758 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb970094e60 con 0x7fb97c0d0e00 2024-04-25T10:59:53.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.904+0000 7fb97affd640 1 --2- 172.21.15.69:0/2136138758 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb9680744e0 0x7fb9680769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:53.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.905+0000 7fb97affd640 1 --2- 172.21.15.69:0/2136138758 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb9680744e0 0x7fb9680769f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0099c0 tx=0x7fb96c003990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:53.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:53.909+0000 7fb9637fe640 1 -- 172.21.15.69:0/2136138758 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb97005f5f0 con 0x7fb97c0d0e00 2024-04-25T10:59:54.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.085+0000 7fb981451640 1 -- 172.21.15.69:0/2136138758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb97c050740 con 0x7fb97c0d0e00 2024-04-25T10:59:54.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.087+0000 7fb9637fe640 1 -- 172.21.15.69:0/2136138758 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7fb97005eef0 con 0x7fb97c0d0e00 2024-04-25T10:59:54.087 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:54.088 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":20,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T10:59:20.630151+0000","last_up_change":"2024-04-25T10:59:05.546839+0000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:59:06.422674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:59:16.727980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:59:17.938486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6825","nonce":1876101065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6827","nonce":1876101065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6831","nonce":1876101065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6829","nonce":1876101065}]},"public_addr":"172.21.15.69:6825/1876101065","cluster_addr":"172.21.15.69:6827/1876101065","heartbeat_back_addr":"172.21.15.69:6831/1876101065","heartbeat_front_addr":"172.21.15.69:6829/1876101065","state":["exists","up"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6801","nonce":1806438563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6803","nonce":1806438563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6807","nonce":1806438563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6805","nonce":1806438563}]},"public_addr":"172.21.15.69:6801/1806438563","cluster_addr":"172.21.15.69:6803/1806438563","heartbeat_back_addr":"172.21.15.69:6807/1806438563","heartbeat_front_addr":"172.21.15.69:6805/1806438563","state":["exists","up"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6817","nonce":1593617983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6819","nonce":1593617983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6823","nonce":1593617983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6821","nonce":1593617983}]},"public_addr":"172.21.15.69:6817/1593617983","cluster_addr":"172.21.15.69:6819/1593617983","heartbeat_back_addr":"172.21.15.69:6823/1593617983","heartbeat_front_addr":"172.21.15.69:6821/1593617983","state":["exists","up"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6809","nonce":3950661294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6811","nonce":3950661294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6815","nonce":3950661294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6813","nonce":3950661294}]},"public_addr":"172.21.15.69:6809/3950661294","cluster_addr":"172.21.15.69:6811/3950661294","heartbeat_back_addr":"172.21.15.69:6815/3950661294","heartbeat_front_addr":"172.21.15.69:6813/3950661294","state":["exists","up"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6801","nonce":2508929872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6803","nonce":2508929872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6807","nonce":2508929872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6805","nonce":2508929872}]},"public_addr":"172.21.15.146:6801/2508929872","cluster_addr":"172.21.15.146:6803/2508929872","heartbeat_back_addr":"172.21.15.146:6807/2508929872","heartbeat_front_addr":"172.21.15.146:6805/2508929872","state":["exists","up"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6811","nonce":1310911138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6816","nonce":1310911138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6823","nonce":1310911138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6821","nonce":1310911138}]},"public_addr":"172.21.15.146:6811/1310911138","cluster_addr":"172.21.15.146:6816/1310911138","heartbeat_back_addr":"172.21.15.146:6823/1310911138","heartbeat_front_addr":"172.21.15.146:6821/1310911138","state":["exists","up"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6810","nonce":634688021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6813","nonce":634688021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6820","nonce":634688021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6817","nonce":634688021}]},"public_addr":"172.21.15.146:6810/634688021","cluster_addr":"172.21.15.146:6813/634688021","heartbeat_back_addr":"172.21.15.146:6820/634688021","heartbeat_front_addr":"172.21.15.146:6817/634688021","state":["exists","up"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6825","nonce":3814215935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6827","nonce":3814215935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6831","nonce":3814215935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6829","nonce":3814215935}]},"public_addr":"172.21.15.146:6825/3814215935","cluster_addr":"172.21.15.146:6827/3814215935","heartbeat_back_addr":"172.21.15.146:6831/3814215935","heartbeat_front_addr":"172.21.15.146:6829/3814215935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.820200+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.850189+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.764560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.866248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.807897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.869881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.782310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.827131+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:59:54.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.089+0000 7fb981451640 1 -- 172.21.15.69:0/2136138758 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb9680744e0 msgr2=0x7fb9680769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:54.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.089+0000 7fb981451640 1 --2- 172.21.15.69:0/2136138758 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb9680744e0 0x7fb9680769f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0099c0 tx=0x7fb96c003990 comp rx=0 tx=0).stop 2024-04-25T10:59:54.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.089+0000 7fb981451640 1 -- 172.21.15.69:0/2136138758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb97c0d0e00 msgr2=0x7fb97c1512e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:54.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.089+0000 7fb981451640 1 --2- 172.21.15.69:0/2136138758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb97c0d0e00 0x7fb97c1512e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb97000c430 tx=0x7fb97000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:59:54.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.089+0000 7fb981451640 1 -- 172.21.15.69:0/2136138758 shutdown_connections 2024-04-25T10:59:54.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.089+0000 7fb981451640 1 --2- 172.21.15.69:0/2136138758 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb9680744e0 0x7fb9680769f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:54.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.089+0000 7fb981451640 1 --2- 172.21.15.69:0/2136138758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb97c0d4a50 0x7fb97c0cba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:54.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.089+0000 7fb981451640 1 --2- 172.21.15.69:0/2136138758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb97c0d0e00 0x7fb97c1512e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:54.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.089+0000 7fb981451640 1 --2- 172.21.15.69:0/2136138758 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb97c06c5c0 0x7fb97c0c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:54.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.089+0000 7fb981451640 1 -- 172.21.15.69:0/2136138758 >> 172.21.15.69:0/2136138758 conn(0x7fb97c0d5d90 msgr2=0x7fb97c15ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:54.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.089+0000 7fb981451640 1 -- 172.21.15.69:0/2136138758 shutdown_connections 2024-04-25T10:59:54.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.089+0000 7fb981451640 1 -- 172.21.15.69:0/2136138758 wait complete. 2024-04-25T10:59:54.101 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:59:06.422674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:59:16.727980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:59:17.938486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:59:54.101 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:59:54.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.210+0000 7f988e4d1640 1 Processor -- start 2024-04-25T10:59:54.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.210+0000 7f988e4d1640 1 -- start start 2024-04-25T10:59:54.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.212+0000 7f988e4d1640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f988806dce0 0x7f98880704c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:54.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.212+0000 7f988e4d1640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f988815adf0 0x7f98880cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:54.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.212+0000 7f988e4d1640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f988806fa20 con 0x7f98880b0c90 2024-04-25T10:59:54.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.212+0000 7f988e4d1640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f9888070fd0 con 0x7f988806dce0 2024-04-25T10:59:54.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.212+0000 7f988e4d1640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f98880712b0 con 0x7f988815adf0 2024-04-25T10:59:54.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.212+0000 7f98877fe640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f988806dce0 0x7f98880704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:54.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.212+0000 7f988ca6e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f988815adf0 0x7f98880cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:54.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.212+0000 7f98877fe640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f988806dce0 0x7f98880704c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.69:41972/0 (socket says 172.21.15.69:41972) 2024-04-25T10:59:54.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.212+0000 7f9887fff640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f98880b0c90 0x7f988806d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.69:48846/0 (socket says 172.21.15.69:48846) 2024-04-25T10:59:54.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.212+0000 7f98877fe640 1 -- 172.21.15.69:0/3414262821 learned_addr learned my addr 172.21.15.69:0/3414262821 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:54.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.212+0000 7f988ca6e640 1 -- 172.21.15.69:0/3414262821 >> v2:172.21.15.146:3300/0 conn(0x7f988806dce0 msgr2=0x7f98880704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:54.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.212+0000 7f988ca6e640 1 --2- 172.21.15.69:0/3414262821 >> v2:172.21.15.146:3300/0 conn(0x7f988806dce0 0x7f98880704c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:54.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.212+0000 7f988ca6e640 1 -- 172.21.15.69:0/3414262821 >> v1:172.21.15.146:6789/0 conn(0x7f98880b0c90 legacy=0x7f988806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:54.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.213+0000 7f988ca6e640 1 -- 172.21.15.69:0/3414262821 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f988806fcc0 con 0x7f988815adf0 2024-04-25T10:59:54.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.213+0000 7f98877fe640 1 --2- 172.21.15.69:0/3414262821 >> v2:172.21.15.146:3300/0 conn(0x7f988806dce0 0x7f98880704c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:59:54.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.213+0000 7f988ca6e640 1 --2- 172.21.15.69:0/3414262821 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f988815adf0 0x7f98880cec50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9880009d90 tx=0x7f988002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b49774bb34867722 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:54.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.213+0000 7f9886ffd640 1 -- 172.21.15.69:0/3414262821 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880007740 con 0x7f988815adf0 2024-04-25T10:59:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.213+0000 7f9886ffd640 1 -- 172.21.15.69:0/3414262821 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98800078e0 con 0x7f988815adf0 2024-04-25T10:59:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.213+0000 7f9886ffd640 1 -- 172.21.15.69:0/3414262821 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880007cb0 con 0x7f988815adf0 2024-04-25T10:59:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.214+0000 7f988e4d1640 1 -- 172.21.15.69:0/3414262821 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f988815adf0 msgr2=0x7f98880cec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.214+0000 7f988e4d1640 1 --2- 172.21.15.69:0/3414262821 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f988815adf0 0x7f98880cec50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9880009d90 tx=0x7f988002a660 comp rx=0 tx=0).stop 2024-04-25T10:59:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.214+0000 7f988e4d1640 1 -- 172.21.15.69:0/3414262821 shutdown_connections 2024-04-25T10:59:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.214+0000 7f988e4d1640 1 --2- 172.21.15.69:0/3414262821 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f988815adf0 0x7f98880cec50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.214+0000 7f988e4d1640 1 --2- 172.21.15.69:0/3414262821 >> v2:172.21.15.146:3300/0 conn(0x7f988806dce0 0x7f98880704c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.214+0000 7f988e4d1640 1 -- 172.21.15.69:0/3414262821 >> 172.21.15.69:0/3414262821 conn(0x7f9888073090 msgr2=0x7f98880734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.214+0000 7f988e4d1640 1 -- 172.21.15.69:0/3414262821 shutdown_connections 2024-04-25T10:59:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.214+0000 7f988e4d1640 1 -- 172.21.15.69:0/3414262821 wait complete. 2024-04-25T10:59:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.216+0000 7f988e4d1640 1 Processor -- start 2024-04-25T10:59:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.216+0000 7f988e4d1640 1 -- start start 2024-04-25T10:59:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.217+0000 7f988e4d1640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98880b0c90 0x7f98881f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.217+0000 7f988e4d1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f988815adf0 0x7f988815fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.217+0000 7f988e4d1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9888149320 0x7f988815c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.217+0000 7f988e4d1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f988815d790 con 0x7f9888149320 2024-04-25T10:59:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.217+0000 7f988e4d1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9888070c80 con 0x7f988815adf0 2024-04-25T10:59:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.217+0000 7f988e4d1640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f988815d910 con 0x7f98880b0c90 2024-04-25T10:59:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.218+0000 7f988ca6e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9888149320 0x7f988815c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.218+0000 7f988ca6e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9888149320 0x7f988815c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55308/0 (socket says 172.21.15.69:55308) 2024-04-25T10:59:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.218+0000 7f988ca6e640 1 -- 172.21.15.69:0/1993294105 learned_addr learned my addr 172.21.15.69:0/1993294105 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.218+0000 7f9887fff640 1 --2- 172.21.15.69:0/1993294105 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98880b0c90 0x7f98881f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.218+0000 7f98877fe640 1 --2- 172.21.15.69:0/1993294105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f988815adf0 0x7f988815fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.218+0000 7f988ca6e640 1 -- 172.21.15.69:0/1993294105 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98880b0c90 msgr2=0x7f98881f4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.218+0000 7f988ca6e640 1 --2- 172.21.15.69:0/1993294105 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98880b0c90 0x7f98881f4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.218+0000 7f988ca6e640 1 -- 172.21.15.69:0/1993294105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f988815adf0 msgr2=0x7f988815fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.218+0000 7f988ca6e640 1 --2- 172.21.15.69:0/1993294105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f988815adf0 0x7f988815fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.218+0000 7f988ca6e640 1 -- 172.21.15.69:0/1993294105 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98881f5290 con 0x7f9888149320 2024-04-25T10:59:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.218+0000 7f988ca6e640 1 --2- 172.21.15.69:0/1993294105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9888149320 0x7f988815c250 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f988002ac50 tx=0x7f9880004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.218+0000 7f9884ff9640 1 -- 172.21.15.69:0/1993294105 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988003f070 con 0x7f9888149320 2024-04-25T10:59:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.218+0000 7f9884ff9640 1 -- 172.21.15.69:0/1993294105 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9880031bc0 con 0x7f9888149320 2024-04-25T10:59:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.218+0000 7f988e4d1640 1 -- 172.21.15.69:0/1993294105 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98881f5570 con 0x7f9888149320 2024-04-25T10:59:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.218+0000 7f9884ff9640 1 -- 172.21.15.69:0/1993294105 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880009450 con 0x7f9888149320 2024-04-25T10:59:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.218+0000 7f988e4d1640 1 -- 172.21.15.69:0/1993294105 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9888149730 con 0x7f9888149320 2024-04-25T10:59:54.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.219+0000 7f988e4d1640 1 -- 172.21.15.69:0/1993294105 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9888050740 con 0x7f9888149320 2024-04-25T10:59:54.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.220+0000 7f9884ff9640 1 -- 172.21.15.69:0/1993294105 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f988003b410 con 0x7f9888149320 2024-04-25T10:59:54.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.220+0000 7f9884ff9640 1 --2- 172.21.15.69:0/1993294105 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f98740744b0 0x7f98740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:54.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.220+0000 7f9884ff9640 1 -- 172.21.15.69:0/1993294105 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f98800b6940 con 0x7f9888149320 2024-04-25T10:59:54.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.220+0000 7f9887fff640 1 --2- 172.21.15.69:0/1993294105 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f98740744b0 0x7f98740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:54.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.221+0000 7f9887fff640 1 --2- 172.21.15.69:0/1993294105 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f98740744b0 0x7f98740769c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9878007fd0 tx=0x7f9878007e80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:54.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.225+0000 7f9884ff9640 1 -- 172.21.15.69:0/1993294105 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9880080a80 con 0x7f9888149320 2024-04-25T10:59:54.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.398+0000 7f988e4d1640 1 -- 172.21.15.69:0/1993294105 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f98880b3260 con 0x7f9888149320 2024-04-25T10:59:54.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.399+0000 7f9884ff9640 1 -- 172.21.15.69:0/1993294105 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f9880084d80 con 0x7f9888149320 2024-04-25T10:59:54.400 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-25T10:59:54.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.401+0000 7f988e4d1640 1 -- 172.21.15.69:0/1993294105 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f98740744b0 msgr2=0x7f98740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:54.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.401+0000 7f988e4d1640 1 --2- 172.21.15.69:0/1993294105 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f98740744b0 0x7f98740769c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9878007fd0 tx=0x7f9878007e80 comp rx=0 tx=0).stop 2024-04-25T10:59:54.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.401+0000 7f988e4d1640 1 -- 172.21.15.69:0/1993294105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9888149320 msgr2=0x7f988815c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:54.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.401+0000 7f988e4d1640 1 --2- 172.21.15.69:0/1993294105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9888149320 0x7f988815c250 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f988002ac50 tx=0x7f9880004a40 comp rx=0 tx=0).stop 2024-04-25T10:59:54.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.401+0000 7f988e4d1640 1 -- 172.21.15.69:0/1993294105 shutdown_connections 2024-04-25T10:59:54.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.401+0000 7f988e4d1640 1 --2- 172.21.15.69:0/1993294105 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f98740744b0 0x7f98740769c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:54.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.401+0000 7f988e4d1640 1 --2- 172.21.15.69:0/1993294105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9888149320 0x7f988815c250 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:54.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.401+0000 7f988e4d1640 1 --2- 172.21.15.69:0/1993294105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f988815adf0 0x7f988815fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:54.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.401+0000 7f988e4d1640 1 --2- 172.21.15.69:0/1993294105 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98880b0c90 0x7f98881f4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:54.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.401+0000 7f988e4d1640 1 -- 172.21.15.69:0/1993294105 >> 172.21.15.69:0/1993294105 conn(0x7f9888073090 msgr2=0x7f9888152bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:54.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.401+0000 7f988e4d1640 1 -- 172.21.15.69:0/1993294105 shutdown_connections 2024-04-25T10:59:54.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.401+0000 7f988e4d1640 1 -- 172.21.15.69:0/1993294105 wait complete. 2024-04-25T10:59:54.413 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:59:54.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.522+0000 7f7f79f7a640 1 Processor -- start 2024-04-25T10:59:54.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.522+0000 7f7f79f7a640 1 -- start start 2024-04-25T10:59:54.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.524+0000 7f7f79f7a640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7f7406dce0 0x7f7f74168f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:54.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.524+0000 7f7f79f7a640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f740cde60 con 0x7f7f740cd9a0 2024-04-25T10:59:54.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.524+0000 7f7f79f7a640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f740ce140 con 0x7f7f740b0c80 2024-04-25T10:59:54.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.524+0000 7f7f79f7a640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f7f740ce420 con 0x7f7f7406dce0 2024-04-25T10:59:54.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.524+0000 7f7f737fe640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f7f740cd9a0 0x7f7f7415f360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:50238/0 (socket says 172.21.15.69:50238) 2024-04-25T10:59:54.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.524+0000 7f7f737fe640 1 -- 172.21.15.69:0/4075918975 learned_addr learned my addr 172.21.15.69:0/4075918975 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:54.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.524+0000 7f7f73fff640 1 --2- 172.21.15.69:0/4075918975 >> v2:172.21.15.69:3300/0 conn(0x7f7f7406dce0 0x7f7f74168f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:54.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.524+0000 7f7f73fff640 1 -- 172.21.15.69:0/4075918975 >> v1:172.21.15.146:6789/0 conn(0x7f7f740b0c80 legacy=0x7f7f7406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:54.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.524+0000 7f7f73fff640 1 -- 172.21.15.69:0/4075918975 >> v1:172.21.15.69:6789/0 conn(0x7f7f740cd9a0 legacy=0x7f7f7415f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:54.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.524+0000 7f7f73fff640 1 -- 172.21.15.69:0/4075918975 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f740ce700 con 0x7f7f7406dce0 2024-04-25T10:59:54.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.524+0000 7f7f727fc640 1 -- 172.21.15.69:0/4075918975 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1659696122 0 0) 0x7f7f740cde60 con 0x7f7f740cd9a0 2024-04-25T10:59:54.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.525+0000 7f7f73fff640 1 --2- 172.21.15.69:0/4075918975 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f7406dce0 0x7f7f74168f10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c009d90 tx=0x7f7f6c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87674897fe031a5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:54.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.525+0000 7f7f727fc640 1 -- 172.21.15.69:0/4075918975 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c007740 con 0x7f7f7406dce0 2024-04-25T10:59:54.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.525+0000 7f7f727fc640 1 -- 172.21.15.69:0/4075918975 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f6c0078e0 con 0x7f7f7406dce0 2024-04-25T10:59:54.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.525+0000 7f7f727fc640 1 -- 172.21.15.69:0/4075918975 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c007cb0 con 0x7f7f7406dce0 2024-04-25T10:59:54.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.526+0000 7f7f79f7a640 1 -- 172.21.15.69:0/4075918975 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f7406dce0 msgr2=0x7f7f74168f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:54.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.526+0000 7f7f79f7a640 1 --2- 172.21.15.69:0/4075918975 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f7406dce0 0x7f7f74168f10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c009d90 tx=0x7f7f6c02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:59:54.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.526+0000 7f7f79f7a640 1 -- 172.21.15.69:0/4075918975 shutdown_connections 2024-04-25T10:59:54.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.526+0000 7f7f79f7a640 1 --2- 172.21.15.69:0/4075918975 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f7406dce0 0x7f7f74168f10 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:54.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.526+0000 7f7f79f7a640 1 -- 172.21.15.69:0/4075918975 >> 172.21.15.69:0/4075918975 conn(0x7f7f74073090 msgr2=0x7f7f740734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:54.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.526+0000 7f7f79f7a640 1 -- 172.21.15.69:0/4075918975 shutdown_connections 2024-04-25T10:59:54.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.526+0000 7f7f79f7a640 1 -- 172.21.15.69:0/4075918975 wait complete. 2024-04-25T10:59:54.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.527+0000 7f7f79f7a640 1 Processor -- start 2024-04-25T10:59:54.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.528+0000 7f7f79f7a640 1 -- start start 2024-04-25T10:59:54.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.529+0000 7f7f79f7a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f740b0c80 0x7f7f7415d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:54.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.529+0000 7f7f79f7a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f7414ea90 0x7f7f74147770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:54.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.529+0000 7f7f79f7a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f7415c500 0x7f7f7415c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:54.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.529+0000 7f7f79f7a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7f740cf080 con 0x7f7f740b0c80 2024-04-25T10:59:54.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.529+0000 7f7f79f7a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7f740cef00 con 0x7f7f7415c500 2024-04-25T10:59:54.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.529+0000 7f7f79f7a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7f74169450 con 0x7f7f7414ea90 2024-04-25T10:59:54.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.529+0000 7f7f737fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f740b0c80 0x7f7f7415d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:54.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.529+0000 7f7f737fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f740b0c80 0x7f7f7415d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55332/0 (socket says 172.21.15.69:55332) 2024-04-25T10:59:54.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.529+0000 7f7f737fe640 1 -- 172.21.15.69:0/1349239732 learned_addr learned my addr 172.21.15.69:0/1349239732 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:54.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.529+0000 7f7f73fff640 1 --2- 172.21.15.69:0/1349239732 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f7415c500 0x7f7f7415c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:54.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.529+0000 7f7f72ffd640 1 --2- 172.21.15.69:0/1349239732 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f7414ea90 0x7f7f74147770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:54.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.529+0000 7f7f737fe640 1 -- 172.21.15.69:0/1349239732 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f7414ea90 msgr2=0x7f7f74147770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:54.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.529+0000 7f7f737fe640 1 --2- 172.21.15.69:0/1349239732 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f7414ea90 0x7f7f74147770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:54.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.529+0000 7f7f737fe640 1 -- 172.21.15.69:0/1349239732 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f7415c500 msgr2=0x7f7f7415c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:54.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.529+0000 7f7f737fe640 1 --2- 172.21.15.69:0/1349239732 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f7415c500 0x7f7f7415c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:54.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.529+0000 7f7f737fe640 1 -- 172.21.15.69:0/1349239732 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f74155f70 con 0x7f7f740b0c80 2024-04-25T10:59:54.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.529+0000 7f7f737fe640 1 --2- 172.21.15.69:0/1349239732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f740b0c80 0x7f7f7415d870 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400c440 tx=0x7f7f6400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:54.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.530+0000 7f7f5bfff640 1 -- 172.21.15.69:0/1349239732 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f64018070 con 0x7f7f740b0c80 2024-04-25T10:59:54.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.530+0000 7f7f5bfff640 1 -- 172.21.15.69:0/1349239732 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f640074e0 con 0x7f7f740b0c80 2024-04-25T10:59:54.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.530+0000 7f7f79f7a640 1 -- 172.21.15.69:0/1349239732 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f7415ce50 con 0x7f7f740b0c80 2024-04-25T10:59:54.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.530+0000 7f7f5bfff640 1 -- 172.21.15.69:0/1349239732 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f64004a20 con 0x7f7f740b0c80 2024-04-25T10:59:54.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.530+0000 7f7f79f7a640 1 -- 172.21.15.69:0/1349239732 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f74152590 con 0x7f7f740b0c80 2024-04-25T10:59:54.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.531+0000 7f7f79f7a640 1 -- 172.21.15.69:0/1349239732 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f74169230 con 0x7f7f740b0c80 2024-04-25T10:59:54.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.531+0000 7f7f5bfff640 1 -- 172.21.15.69:0/1349239732 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7f6401b070 con 0x7f7f740b0c80 2024-04-25T10:59:54.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.532+0000 7f7f5bfff640 1 --2- 172.21.15.69:0/1349239732 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f7f600744b0 0x7f7f600769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:54.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.532+0000 7f7f5bfff640 1 -- 172.21.15.69:0/1349239732 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7f64094500 con 0x7f7f740b0c80 2024-04-25T10:59:54.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.532+0000 7f7f72ffd640 1 --2- 172.21.15.69:0/1349239732 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f7f600744b0 0x7f7f600769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:54.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.532+0000 7f7f72ffd640 1 --2- 172.21.15.69:0/1349239732 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f7f600744b0 0x7f7f600769c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7f68006fd0 tx=0x7f7f68008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:54.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.536+0000 7f7f5bfff640 1 -- 172.21.15.69:0/1349239732 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7f7415ce50 con 0x7f7f740b0c80 2024-04-25T10:59:54.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.713+0000 7f7f79f7a640 1 -- 172.21.15.69:0/1349239732 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7f740b0540 con 0x7f7f740b0c80 2024-04-25T10:59:54.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.714+0000 7f7f5bfff640 1 -- 172.21.15.69:0/1349239732 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f7f74155f70 con 0x7f7f740b0c80 2024-04-25T10:59:54.715 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-25T10:59:54.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.716+0000 7f7f79f7a640 1 -- 172.21.15.69:0/1349239732 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f7f600744b0 msgr2=0x7f7f600769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:54.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.716+0000 7f7f79f7a640 1 --2- 172.21.15.69:0/1349239732 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f7f600744b0 0x7f7f600769c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7f68006fd0 tx=0x7f7f68008040 comp rx=0 tx=0).stop 2024-04-25T10:59:54.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.716+0000 7f7f79f7a640 1 -- 172.21.15.69:0/1349239732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f740b0c80 msgr2=0x7f7f7415d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:54.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.716+0000 7f7f79f7a640 1 --2- 172.21.15.69:0/1349239732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f740b0c80 0x7f7f7415d870 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400c440 tx=0x7f7f6400c900 comp rx=0 tx=0).stop 2024-04-25T10:59:54.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.716+0000 7f7f79f7a640 1 -- 172.21.15.69:0/1349239732 shutdown_connections 2024-04-25T10:59:54.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.716+0000 7f7f79f7a640 1 --2- 172.21.15.69:0/1349239732 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f7f600744b0 0x7f7f600769c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:54.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.716+0000 7f7f79f7a640 1 --2- 172.21.15.69:0/1349239732 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f7415c500 0x7f7f7415c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:54.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.716+0000 7f7f79f7a640 1 --2- 172.21.15.69:0/1349239732 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f7414ea90 0x7f7f74147770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:54.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.716+0000 7f7f79f7a640 1 --2- 172.21.15.69:0/1349239732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f740b0c80 0x7f7f7415d870 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:54.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.716+0000 7f7f79f7a640 1 -- 172.21.15.69:0/1349239732 >> 172.21.15.69:0/1349239732 conn(0x7f7f74073090 msgr2=0x7f7f740b1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:54.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.716+0000 7f7f79f7a640 1 -- 172.21.15.69:0/1349239732 shutdown_connections 2024-04-25T10:59:54.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.716+0000 7f7f79f7a640 1 -- 172.21.15.69:0/1349239732 wait complete. 2024-04-25T10:59:54.727 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:59:54.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.837+0000 7f5e340e9640 1 Processor -- start 2024-04-25T10:59:54.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.837+0000 7f5e340e9640 1 -- start start 2024-04-25T10:59:54.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.838+0000 7f5e340e9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e2c0ce250 0x7f5e2c15cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:54.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.839+0000 7f5e340e9640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f5e2c06d5d0 0x7f5e2c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:54.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.839+0000 7f5e340e9640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e2c06df20 con 0x7f5e2c1458c0 2024-04-25T10:59:54.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.839+0000 7f5e340e9640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f5e2c0b4ec0 con 0x7f5e2c06d5d0 2024-04-25T10:59:54.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.839+0000 7f5e340e9640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5e2c145cd0 con 0x7f5e2c0ce250 2024-04-25T10:59:54.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.839+0000 7f5e32686640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f5e2c06d5d0 0x7f5e2c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:54.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.839+0000 7f5e32686640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f5e2c06d5d0 0x7f5e2c06d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55336/0 (socket says 172.21.15.69:55336) 2024-04-25T10:59:54.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.839+0000 7f5e31684640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f5e2c1458c0 0x7f5e2c0b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:50242/0 (socket says 172.21.15.69:50242) 2024-04-25T10:59:54.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.839+0000 7f5e32686640 1 -- 172.21.15.69:0/673691397 learned_addr learned my addr 172.21.15.69:0/673691397 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:54.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.839+0000 7f5e31e85640 1 --2- 172.21.15.69:0/673691397 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e2c0ce250 0x7f5e2c15cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:54.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.839+0000 7f5e32686640 1 -- 172.21.15.69:0/673691397 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e2c0ce250 msgr2=0x7f5e2c15cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:54.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.839+0000 7f5e32686640 1 --2- 172.21.15.69:0/673691397 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e2c0ce250 0x7f5e2c15cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:54.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.839+0000 7f5e32686640 1 -- 172.21.15.69:0/673691397 >> v1:172.21.15.69:6789/0 conn(0x7f5e2c1458c0 legacy=0x7f5e2c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:54.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.839+0000 7f5e32686640 1 -- 172.21.15.69:0/673691397 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e2c15d5c0 con 0x7f5e2c06d5d0 2024-04-25T10:59:54.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.839+0000 7f5e30e83640 1 -- 172.21.15.69:0/673691397 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4207482752 0 0) 0x7f5e1c001110 con 0x7f5e2c1458c0 2024-04-25T10:59:54.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.840+0000 7f5e32686640 1 --2- 172.21.15.69:0/673691397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e2c06d5d0 0x7f5e2c06d9e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009870 tx=0x7f5e2402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8ec92bde3859ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:54.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.840+0000 7f5e30e83640 1 -- 172.21.15.69:0/673691397 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e24037070 con 0x7f5e2c06d5d0 2024-04-25T10:59:54.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.840+0000 7f5e30e83640 1 -- 172.21.15.69:0/673691397 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e24032440 con 0x7f5e2c06d5d0 2024-04-25T10:59:54.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.840+0000 7f5e30e83640 1 -- 172.21.15.69:0/673691397 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e24032810 con 0x7f5e2c06d5d0 2024-04-25T10:59:54.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.841+0000 7f5e340e9640 1 -- 172.21.15.69:0/673691397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e2c06d5d0 msgr2=0x7f5e2c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:54.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.841+0000 7f5e340e9640 1 --2- 172.21.15.69:0/673691397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e2c06d5d0 0x7f5e2c06d9e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009870 tx=0x7f5e2402a1c0 comp rx=0 tx=0).stop 2024-04-25T10:59:54.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.841+0000 7f5e340e9640 1 -- 172.21.15.69:0/673691397 shutdown_connections 2024-04-25T10:59:54.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.841+0000 7f5e340e9640 1 --2- 172.21.15.69:0/673691397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e2c06d5d0 0x7f5e2c06d9e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:54.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.841+0000 7f5e340e9640 1 --2- 172.21.15.69:0/673691397 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e2c0ce250 0x7f5e2c15cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:54.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.841+0000 7f5e340e9640 1 -- 172.21.15.69:0/673691397 >> 172.21.15.69:0/673691397 conn(0x7f5e2c073090 msgr2=0x7f5e2c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:54.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.841+0000 7f5e340e9640 1 -- 172.21.15.69:0/673691397 shutdown_connections 2024-04-25T10:59:54.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.841+0000 7f5e340e9640 1 -- 172.21.15.69:0/673691397 wait complete. 2024-04-25T10:59:54.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.842+0000 7f5e340e9640 1 Processor -- start 2024-04-25T10:59:54.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.842+0000 7f5e340e9640 1 -- start start 2024-04-25T10:59:54.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.844+0000 7f5e340e9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e2c06d5d0 0x7f5e2c144880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:54.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.844+0000 7f5e340e9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e2c0ce250 0x7f5e2c1f5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:54.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.844+0000 7f5e340e9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e2c1458c0 0x7f5e2c137af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:54.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.844+0000 7f5e340e9640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c06f640 con 0x7f5e2c0ce250 2024-04-25T10:59:54.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.844+0000 7f5e340e9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c15db80 con 0x7f5e2c1458c0 2024-04-25T10:59:54.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.844+0000 7f5e340e9640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5e2c15de60 con 0x7f5e2c06d5d0 2024-04-25T10:59:54.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.844+0000 7f5e31684640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e2c0ce250 0x7f5e2c1f5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:54.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.844+0000 7f5e31684640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e2c0ce250 0x7f5e2c1f5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55346/0 (socket says 172.21.15.69:55346) 2024-04-25T10:59:54.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.844+0000 7f5e31684640 1 -- 172.21.15.69:0/2817886019 learned_addr learned my addr 172.21.15.69:0/2817886019 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:54.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.844+0000 7f5e31e85640 1 --2- 172.21.15.69:0/2817886019 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e2c06d5d0 0x7f5e2c144880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:54.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.844+0000 7f5e32686640 1 --2- 172.21.15.69:0/2817886019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e2c1458c0 0x7f5e2c137af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:54.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.844+0000 7f5e31684640 1 -- 172.21.15.69:0/2817886019 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e2c06d5d0 msgr2=0x7f5e2c144880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:54.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.844+0000 7f5e31684640 1 --2- 172.21.15.69:0/2817886019 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e2c06d5d0 0x7f5e2c144880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:54.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.844+0000 7f5e31684640 1 -- 172.21.15.69:0/2817886019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e2c1458c0 msgr2=0x7f5e2c137af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:54.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.844+0000 7f5e31684640 1 --2- 172.21.15.69:0/2817886019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e2c1458c0 0x7f5e2c137af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:54.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.844+0000 7f5e31684640 1 -- 172.21.15.69:0/2817886019 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e2c15c010 con 0x7f5e2c0ce250 2024-04-25T10:59:54.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.844+0000 7f5e31e85640 1 --2- 172.21.15.69:0/2817886019 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e2c06d5d0 0x7f5e2c144880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:59:54.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.844+0000 7f5e31684640 1 --2- 172.21.15.69:0/2817886019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e2c0ce250 0x7f5e2c1f5cd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00c430 tx=0x7f5e1c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:54.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.844+0000 7f5e227fc640 1 -- 172.21.15.69:0/2817886019 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c0074e0 con 0x7f5e2c0ce250 2024-04-25T10:59:54.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.844+0000 7f5e227fc640 1 -- 172.21.15.69:0/2817886019 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e1c0047e0 con 0x7f5e2c0ce250 2024-04-25T10:59:54.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.844+0000 7f5e340e9640 1 -- 172.21.15.69:0/2817886019 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e2c1f68e0 con 0x7f5e2c0ce250 2024-04-25T10:59:54.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.844+0000 7f5e227fc640 1 -- 172.21.15.69:0/2817886019 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c004bd0 con 0x7f5e2c0ce250 2024-04-25T10:59:54.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.844+0000 7f5e340e9640 1 -- 172.21.15.69:0/2817886019 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e2c143da0 con 0x7f5e2c0ce250 2024-04-25T10:59:54.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.845+0000 7f5e340e9640 1 -- 172.21.15.69:0/2817886019 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e2c1f6650 con 0x7f5e2c0ce250 2024-04-25T10:59:54.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.846+0000 7f5e227fc640 1 -- 172.21.15.69:0/2817886019 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5e1c011070 con 0x7f5e2c0ce250 2024-04-25T10:59:54.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.847+0000 7f5e227fc640 1 --2- 172.21.15.69:0/2817886019 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5e100744b0 0x7f5e100769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:54.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.847+0000 7f5e31e85640 1 --2- 172.21.15.69:0/2817886019 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5e100744b0 0x7f5e100769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:54.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.847+0000 7f5e227fc640 1 -- 172.21.15.69:0/2817886019 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5e1c094ea0 con 0x7f5e2c0ce250 2024-04-25T10:59:54.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.847+0000 7f5e31e85640 1 --2- 172.21.15.69:0/2817886019 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5e100744b0 0x7f5e100769c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c0b6c80 tx=0x7f5e280039d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:54.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:54.852+0000 7f5e227fc640 1 -- 172.21.15.69:0/2817886019 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5e2c15c010 con 0x7f5e2c0ce250 2024-04-25T10:59:55.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.026+0000 7f5e340e9640 1 -- 172.21.15.69:0/2817886019 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5e2c144570 con 0x7f5e2c0ce250 2024-04-25T10:59:55.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.026+0000 7f5e227fc640 1 -- 172.21.15.69:0/2817886019 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f5e2c06df20 con 0x7f5e2c0ce250 2024-04-25T10:59:55.027 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-25T10:59:55.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.028+0000 7f5e340e9640 1 -- 172.21.15.69:0/2817886019 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5e100744b0 msgr2=0x7f5e100769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.028+0000 7f5e340e9640 1 --2- 172.21.15.69:0/2817886019 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5e100744b0 0x7f5e100769c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c0b6c80 tx=0x7f5e280039d0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.028+0000 7f5e340e9640 1 -- 172.21.15.69:0/2817886019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e2c0ce250 msgr2=0x7f5e2c1f5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.028+0000 7f5e340e9640 1 --2- 172.21.15.69:0/2817886019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e2c0ce250 0x7f5e2c1f5cd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00c430 tx=0x7f5e1c00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.028+0000 7f5e340e9640 1 -- 172.21.15.69:0/2817886019 shutdown_connections 2024-04-25T10:59:55.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.028+0000 7f5e340e9640 1 --2- 172.21.15.69:0/2817886019 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5e100744b0 0x7f5e100769c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.028+0000 7f5e340e9640 1 --2- 172.21.15.69:0/2817886019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e2c1458c0 0x7f5e2c137af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.028+0000 7f5e340e9640 1 --2- 172.21.15.69:0/2817886019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e2c0ce250 0x7f5e2c1f5cd0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.028+0000 7f5e340e9640 1 --2- 172.21.15.69:0/2817886019 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e2c06d5d0 0x7f5e2c144880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.028+0000 7f5e340e9640 1 -- 172.21.15.69:0/2817886019 >> 172.21.15.69:0/2817886019 conn(0x7f5e2c073090 msgr2=0x7f5e2c0b5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:55.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.028+0000 7f5e340e9640 1 -- 172.21.15.69:0/2817886019 shutdown_connections 2024-04-25T10:59:55.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.028+0000 7f5e340e9640 1 -- 172.21.15.69:0/2817886019 wait complete. 2024-04-25T10:59:55.040 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:59:55.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.146+0000 7fc667067640 1 Processor -- start 2024-04-25T10:59:55.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.146+0000 7fc667067640 1 -- start start 2024-04-25T10:59:55.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.148+0000 7fc667067640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc6600b7750 0x7fc6600d1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.148+0000 7fc667067640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6600cf610 0x7fc660164800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.148+0000 7fc667067640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fc660164d40 0x7fc66006d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.148+0000 7fc667067640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fc6600b3af0 con 0x7fc660164d40 2024-04-25T10:59:55.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.148+0000 7fc667067640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fc660164320 con 0x7fc6600b7750 2024-04-25T10:59:55.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.148+0000 7fc667067640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc6600b2f70 con 0x7fc6600cf610 2024-04-25T10:59:55.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.148+0000 7fc65ffff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6600cf610 0x7fc660164800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.148+0000 7fc665604640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fc660164d40 0x7fc66006d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.148+0000 7fc65ffff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6600cf610 0x7fc660164800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.69:54880/0 (socket says 172.21.15.69:54880) 2024-04-25T10:59:55.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.148+0000 7fc65ffff640 1 -- 172.21.15.69:0/2393037507 learned_addr learned my addr 172.21.15.69:0/2393037507 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:55.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.148+0000 7fc664e03640 1 --2- 172.21.15.69:0/2393037507 >> v2:172.21.15.146:3300/0 conn(0x7fc6600b7750 0x7fc6600d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.148+0000 7fc665604640 1 -- 172.21.15.69:0/2393037507 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6600cf610 msgr2=0x7fc660164800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:55.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.148+0000 7fc665604640 1 --2- 172.21.15.69:0/2393037507 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6600cf610 0x7fc660164800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.148+0000 7fc665604640 1 -- 172.21.15.69:0/2393037507 >> v2:172.21.15.146:3300/0 conn(0x7fc6600b7750 msgr2=0x7fc6600d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:55.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.148+0000 7fc665604640 1 --2- 172.21.15.69:0/2393037507 >> v2:172.21.15.146:3300/0 conn(0x7fc6600b7750 0x7fc6600d1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.148+0000 7fc665604640 1 -- 172.21.15.69:0/2393037507 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6600b3250 con 0x7fc660164d40 2024-04-25T10:59:55.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.148+0000 7fc65ffff640 1 --2- 172.21.15.69:0/2393037507 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6600cf610 0x7fc660164800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:59:55.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.149+0000 7fc665604640 1 --2- 172.21.15.69:0/2393037507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc660164d40 0x7fc66006d7e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc660073180 tx=0x7fc65802a540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e131dfa5a01cf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:55.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.149+0000 7fc65f7fe640 1 -- 172.21.15.69:0/2393037507 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc658007740 con 0x7fc660164d40 2024-04-25T10:59:55.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.149+0000 7fc65f7fe640 1 -- 172.21.15.69:0/2393037507 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6580078e0 con 0x7fc660164d40 2024-04-25T10:59:55.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.149+0000 7fc65f7fe640 1 -- 172.21.15.69:0/2393037507 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc658007cb0 con 0x7fc660164d40 2024-04-25T10:59:55.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.150+0000 7fc667067640 1 -- 172.21.15.69:0/2393037507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc660164d40 msgr2=0x7fc66006d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.150+0000 7fc667067640 1 --2- 172.21.15.69:0/2393037507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc660164d40 0x7fc66006d7e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc660073180 tx=0x7fc65802a540 comp rx=0 tx=0).stop 2024-04-25T10:59:55.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.150+0000 7fc667067640 1 -- 172.21.15.69:0/2393037507 shutdown_connections 2024-04-25T10:59:55.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.150+0000 7fc667067640 1 --2- 172.21.15.69:0/2393037507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc660164d40 0x7fc66006d7e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.150+0000 7fc667067640 1 --2- 172.21.15.69:0/2393037507 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6600cf610 0x7fc660164800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.150+0000 7fc667067640 1 --2- 172.21.15.69:0/2393037507 >> v2:172.21.15.146:3300/0 conn(0x7fc6600b7750 0x7fc6600d1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.150+0000 7fc667067640 1 -- 172.21.15.69:0/2393037507 >> 172.21.15.69:0/2393037507 conn(0x7fc66009b7d0 msgr2=0x7fc6600b7040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:55.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.150+0000 7fc667067640 1 -- 172.21.15.69:0/2393037507 shutdown_connections 2024-04-25T10:59:55.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.150+0000 7fc667067640 1 -- 172.21.15.69:0/2393037507 wait complete. 2024-04-25T10:59:55.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.152+0000 7fc667067640 1 Processor -- start 2024-04-25T10:59:55.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.152+0000 7fc667067640 1 -- start start 2024-04-25T10:59:55.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.153+0000 7fc667067640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6600b7750 0x7fc6600b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.153+0000 7fc667067640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6600cf610 0x7fc6601541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.153+0000 7fc667067640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc660164d40 0x7fc660162e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.153+0000 7fc667067640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc6600cfa20 con 0x7fc6600cf610 2024-04-25T10:59:55.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.153+0000 7fc667067640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc6600b3810 con 0x7fc6600b7750 2024-04-25T10:59:55.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.153+0000 7fc667067640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc66006dd20 con 0x7fc660164d40 2024-04-25T10:59:55.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.153+0000 7fc65ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6600cf610 0x7fc6601541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.153+0000 7fc65ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6600cf610 0x7fc6601541d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55358/0 (socket says 172.21.15.69:55358) 2024-04-25T10:59:55.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.153+0000 7fc65ffff640 1 -- 172.21.15.69:0/2126703397 learned_addr learned my addr 172.21.15.69:0/2126703397 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:55.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.154+0000 7fc665604640 1 --2- 172.21.15.69:0/2126703397 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc660164d40 0x7fc660162e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.154+0000 7fc664e03640 1 --2- 172.21.15.69:0/2126703397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6600b7750 0x7fc6600b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.154+0000 7fc65ffff640 1 -- 172.21.15.69:0/2126703397 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc660164d40 msgr2=0x7fc660162e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.154+0000 7fc65ffff640 1 --2- 172.21.15.69:0/2126703397 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc660164d40 0x7fc660162e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.154+0000 7fc65ffff640 1 -- 172.21.15.69:0/2126703397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6600b7750 msgr2=0x7fc6600b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.154+0000 7fc65ffff640 1 --2- 172.21.15.69:0/2126703397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6600b7750 0x7fc6600b0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.154+0000 7fc65ffff640 1 -- 172.21.15.69:0/2126703397 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc66015f650 con 0x7fc6600cf610 2024-04-25T10:59:55.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.154+0000 7fc65ffff640 1 --2- 172.21.15.69:0/2126703397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6600cf610 0x7fc6601541d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc65400e9e0 tx=0x7fc65400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:55.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.154+0000 7fc65d7fa640 1 -- 172.21.15.69:0/2126703397 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc654017070 con 0x7fc6600cf610 2024-04-25T10:59:55.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.154+0000 7fc65d7fa640 1 -- 172.21.15.69:0/2126703397 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc65400c9d0 con 0x7fc6600cf610 2024-04-25T10:59:55.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.154+0000 7fc667067640 1 -- 172.21.15.69:0/2126703397 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc66015df80 con 0x7fc6600cf610 2024-04-25T10:59:55.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.154+0000 7fc65d7fa640 1 -- 172.21.15.69:0/2126703397 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65400cd80 con 0x7fc6600cf610 2024-04-25T10:59:55.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.154+0000 7fc667067640 1 -- 172.21.15.69:0/2126703397 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc66015e540 con 0x7fc6600cf610 2024-04-25T10:59:55.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.155+0000 7fc667067640 1 -- 172.21.15.69:0/2126703397 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc660050740 con 0x7fc6600cf610 2024-04-25T10:59:55.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.156+0000 7fc65d7fa640 1 -- 172.21.15.69:0/2126703397 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc65401a070 con 0x7fc6600cf610 2024-04-25T10:59:55.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.156+0000 7fc65d7fa640 1 --2- 172.21.15.69:0/2126703397 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc64c0744b0 0x7fc64c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.156+0000 7fc65d7fa640 1 -- 172.21.15.69:0/2126703397 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc654095440 con 0x7fc6600cf610 2024-04-25T10:59:55.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.156+0000 7fc664e03640 1 --2- 172.21.15.69:0/2126703397 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc64c0744b0 0x7fc64c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.157+0000 7fc664e03640 1 --2- 172.21.15.69:0/2126703397 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc64c0744b0 0x7fc64c0769c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc6600b4dc0 tx=0x7fc650008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:55.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.161+0000 7fc65d7fa640 1 -- 172.21.15.69:0/2126703397 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc66015f650 con 0x7fc6600cf610 2024-04-25T10:59:55.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.372+0000 7fc667067640 1 -- 172.21.15.69:0/2126703397 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6600d2f00 con 0x7fc6600cf610 2024-04-25T10:59:55.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.373+0000 7fc65d7fa640 1 -- 172.21.15.69:0/2126703397 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5178 (secure 0 0 0) 0x7fc65405fc60 con 0x7fc6600cf610 2024-04-25T10:59:55.374 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:55.374 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/2361834023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2361834023},{"type":"v1","addr":"172.21.15.146:6833","nonce":2361834023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1848276963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1848276963},{"type":"v1","addr":"172.21.15.69:6837","nonce":1848276963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1826742846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1826742846},{"type":"v1","addr":"172.21.15.69:6836","nonce":1826742846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:59:18.976232+0000","modified":"2024-04-25T10:59:20.983284+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6835/908008797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":908008797},{"type":"v1","addr":"172.21.15.146:6835","nonce":908008797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-04-25T10:59:55.374 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 6 2024-04-25T10:59:55.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.375+0000 7fc667067640 1 -- 172.21.15.69:0/2126703397 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc64c0744b0 msgr2=0x7fc64c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.375+0000 7fc667067640 1 --2- 172.21.15.69:0/2126703397 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc64c0744b0 0x7fc64c0769c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc6600b4dc0 tx=0x7fc650008040 comp rx=0 tx=0).stop 2024-04-25T10:59:55.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.375+0000 7fc667067640 1 -- 172.21.15.69:0/2126703397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6600cf610 msgr2=0x7fc6601541d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.375+0000 7fc667067640 1 --2- 172.21.15.69:0/2126703397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6600cf610 0x7fc6601541d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc65400e9e0 tx=0x7fc65400eea0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.375+0000 7fc667067640 1 -- 172.21.15.69:0/2126703397 shutdown_connections 2024-04-25T10:59:55.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.375+0000 7fc667067640 1 --2- 172.21.15.69:0/2126703397 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc64c0744b0 0x7fc64c0769c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.375+0000 7fc667067640 1 --2- 172.21.15.69:0/2126703397 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc660164d40 0x7fc660162e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.375+0000 7fc667067640 1 --2- 172.21.15.69:0/2126703397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6600cf610 0x7fc6601541d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.375+0000 7fc667067640 1 --2- 172.21.15.69:0/2126703397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6600b7750 0x7fc6600b0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.375+0000 7fc667067640 1 -- 172.21.15.69:0/2126703397 >> 172.21.15.69:0/2126703397 conn(0x7fc66009b7d0 msgr2=0x7fc66006c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:55.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.375+0000 7fc667067640 1 -- 172.21.15.69:0/2126703397 shutdown_connections 2024-04-25T10:59:55.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.375+0000 7fc667067640 1 -- 172.21.15.69:0/2126703397 wait complete. 2024-04-25T10:59:55.387 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:59:55.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.494+0000 7f5b99306640 1 Processor -- start 2024-04-25T10:59:55.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.494+0000 7f5b99306640 1 -- start start 2024-04-25T10:59:55.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.496+0000 7f5b99306640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5b940d0e00 0x7f5b940d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.496+0000 7f5b99306640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5b940d4a50 0x7f5b940d4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.496+0000 7f5b99306640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b940d1850 con 0x7f5b9406c5c0 2024-04-25T10:59:55.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.496+0000 7f5b99306640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f5b94160de0 con 0x7f5b940d0e00 2024-04-25T10:59:55.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.496+0000 7f5b99306640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5b941610c0 con 0x7f5b940d4a50 2024-04-25T10:59:55.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.496+0000 7f5b937fe640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5b940d4a50 0x7f5b940d4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.496+0000 7f5b92ffd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5b940d0e00 0x7f5b940d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.496+0000 7f5b937fe640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5b940d4a50 0x7f5b940d4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.69:37950/0 (socket says 172.21.15.69:37950) 2024-04-25T10:59:55.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.496+0000 7f5b937fe640 1 -- 172.21.15.69:0/3379335130 learned_addr learned my addr 172.21.15.69:0/3379335130 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:55.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.496+0000 7f5b937fe640 1 -- 172.21.15.69:0/3379335130 >> v2:172.21.15.146:3300/0 conn(0x7f5b940d0e00 msgr2=0x7f5b940d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:55.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.496+0000 7f5b937fe640 1 --2- 172.21.15.69:0/3379335130 >> v2:172.21.15.146:3300/0 conn(0x7f5b940d0e00 0x7f5b940d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.496+0000 7f5b937fe640 1 -- 172.21.15.69:0/3379335130 >> v1:172.21.15.146:6789/0 conn(0x7f5b9406c5c0 legacy=0x7f5b9406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.496+0000 7f5b937fe640 1 -- 172.21.15.69:0/3379335130 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b941613a0 con 0x7f5b940d4a50 2024-04-25T10:59:55.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.497+0000 7f5b937fe640 1 --2- 172.21.15.69:0/3379335130 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5b940d4a50 0x7f5b940d4e60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009e50 tx=0x7f5b8c02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77d7fd522b123486 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:55.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.497+0000 7f5b927fc640 1 -- 172.21.15.69:0/3379335130 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c036070 con 0x7f5b940d4a50 2024-04-25T10:59:55.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.497+0000 7f5b927fc640 1 -- 172.21.15.69:0/3379335130 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b8c031600 con 0x7f5b940d4a50 2024-04-25T10:59:55.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.497+0000 7f5b927fc640 1 -- 172.21.15.69:0/3379335130 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c0319d0 con 0x7f5b940d4a50 2024-04-25T10:59:55.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.498+0000 7f5b99306640 1 -- 172.21.15.69:0/3379335130 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5b940d4a50 msgr2=0x7f5b940d4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.498+0000 7f5b99306640 1 --2- 172.21.15.69:0/3379335130 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5b940d4a50 0x7f5b940d4e60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009e50 tx=0x7f5b8c02a710 comp rx=0 tx=0).stop 2024-04-25T10:59:55.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.498+0000 7f5b99306640 1 -- 172.21.15.69:0/3379335130 shutdown_connections 2024-04-25T10:59:55.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.498+0000 7f5b99306640 1 --2- 172.21.15.69:0/3379335130 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5b940d4a50 0x7f5b940d4e60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.498+0000 7f5b99306640 1 --2- 172.21.15.69:0/3379335130 >> v2:172.21.15.146:3300/0 conn(0x7f5b940d0e00 0x7f5b940d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.498+0000 7f5b99306640 1 -- 172.21.15.69:0/3379335130 >> 172.21.15.69:0/3379335130 conn(0x7f5b940d5d90 msgr2=0x7f5b940d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:55.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.498+0000 7f5b99306640 1 -- 172.21.15.69:0/3379335130 shutdown_connections 2024-04-25T10:59:55.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.498+0000 7f5b99306640 1 -- 172.21.15.69:0/3379335130 wait complete. 2024-04-25T10:59:55.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.500+0000 7f5b99306640 1 Processor -- start 2024-04-25T10:59:55.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.500+0000 7f5b99306640 1 -- start start 2024-04-25T10:59:55.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.501+0000 7f5b99306640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b9406c5c0 0x7f5b940ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.501+0000 7f5b99306640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5b940d0e00 0x7f5b94154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.501+0000 7f5b99306640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b940d4a50 0x7f5b940c9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.501+0000 7f5b99306640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5b94164e00 con 0x7f5b9406c5c0 2024-04-25T10:59:55.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.501+0000 7f5b99306640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5b940b5320 con 0x7f5b940d4a50 2024-04-25T10:59:55.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.501+0000 7f5b99306640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5b94164f80 con 0x7f5b940d0e00 2024-04-25T10:59:55.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.501+0000 7f5b92ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b9406c5c0 0x7f5b940ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.501+0000 7f5b92ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b9406c5c0 0x7f5b940ccdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47990/0 (socket says 172.21.15.69:47990) 2024-04-25T10:59:55.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.501+0000 7f5b92ffd640 1 -- 172.21.15.69:0/3476688721 learned_addr learned my addr 172.21.15.69:0/3476688721 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:55.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.501+0000 7f5b8a7fc640 1 --2- 172.21.15.69:0/3476688721 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5b940d0e00 0x7f5b94154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.501+0000 7f5b937fe640 1 --2- 172.21.15.69:0/3476688721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b940d4a50 0x7f5b940c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.502+0000 7f5b92ffd640 1 -- 172.21.15.69:0/3476688721 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5b940d0e00 msgr2=0x7f5b94154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.502+0000 7f5b92ffd640 1 --2- 172.21.15.69:0/3476688721 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5b940d0e00 0x7f5b94154b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.502+0000 7f5b92ffd640 1 -- 172.21.15.69:0/3476688721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b940d4a50 msgr2=0x7f5b940c9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.502+0000 7f5b92ffd640 1 --2- 172.21.15.69:0/3476688721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b940d4a50 0x7f5b940c9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.502+0000 7f5b92ffd640 1 -- 172.21.15.69:0/3476688721 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b94155050 con 0x7f5b9406c5c0 2024-04-25T10:59:55.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.502+0000 7f5b92ffd640 1 --2- 172.21.15.69:0/3476688721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b9406c5c0 0x7f5b940ccdd0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5b8400e8a0 tx=0x7f5b8400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:55.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.502+0000 7f5b8bfff640 1 -- 172.21.15.69:0/3476688721 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b84017070 con 0x7f5b9406c5c0 2024-04-25T10:59:55.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.502+0000 7f5b8bfff640 1 -- 172.21.15.69:0/3476688721 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b8400c940 con 0x7f5b9406c5c0 2024-04-25T10:59:55.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.502+0000 7f5b99306640 1 -- 172.21.15.69:0/3476688721 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b940cd6d0 con 0x7f5b9406c5c0 2024-04-25T10:59:55.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.502+0000 7f5b8bfff640 1 -- 172.21.15.69:0/3476688721 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8400cd30 con 0x7f5b9406c5c0 2024-04-25T10:59:55.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.502+0000 7f5b99306640 1 -- 172.21.15.69:0/3476688721 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b940cdc90 con 0x7f5b9406c5c0 2024-04-25T10:59:55.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.503+0000 7f5b99306640 1 -- 172.21.15.69:0/3476688721 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b94050740 con 0x7f5b9406c5c0 2024-04-25T10:59:55.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.504+0000 7f5b8bfff640 1 -- 172.21.15.69:0/3476688721 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5b8401a070 con 0x7f5b9406c5c0 2024-04-25T10:59:55.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.504+0000 7f5b8bfff640 1 --2- 172.21.15.69:0/3476688721 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5b740744e0 0x7f5b740769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.504+0000 7f5b8bfff640 1 -- 172.21.15.69:0/3476688721 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5b84095060 con 0x7f5b9406c5c0 2024-04-25T10:59:55.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.504+0000 7f5b8a7fc640 1 --2- 172.21.15.69:0/3476688721 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5b740744e0 0x7f5b740769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.505+0000 7f5b8a7fc640 1 --2- 172.21.15.69:0/3476688721 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5b740744e0 0x7f5b740769f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5b941fa070 tx=0x7f5b7c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:55.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.509+0000 7f5b8bfff640 1 -- 172.21.15.69:0/3476688721 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5b8405f880 con 0x7f5b9406c5c0 2024-04-25T10:59:55.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.684+0000 7f5b99306640 1 -- 172.21.15.69:0/3476688721 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b940d4320 con 0x7f5b9406c5c0 2024-04-25T10:59:55.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.685+0000 7f5b8bfff640 1 -- 172.21.15.69:0/3476688721 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f5b8405f180 con 0x7f5b9406c5c0 2024-04-25T10:59:55.686 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:55.686 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":20,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T10:59:20.630151+0000","last_up_change":"2024-04-25T10:59:05.546839+0000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:59:06.422674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:59:16.727980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:59:17.938486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6825","nonce":1876101065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6827","nonce":1876101065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6831","nonce":1876101065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6829","nonce":1876101065}]},"public_addr":"172.21.15.69:6825/1876101065","cluster_addr":"172.21.15.69:6827/1876101065","heartbeat_back_addr":"172.21.15.69:6831/1876101065","heartbeat_front_addr":"172.21.15.69:6829/1876101065","state":["exists","up"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6801","nonce":1806438563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6803","nonce":1806438563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6807","nonce":1806438563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6805","nonce":1806438563}]},"public_addr":"172.21.15.69:6801/1806438563","cluster_addr":"172.21.15.69:6803/1806438563","heartbeat_back_addr":"172.21.15.69:6807/1806438563","heartbeat_front_addr":"172.21.15.69:6805/1806438563","state":["exists","up"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6817","nonce":1593617983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6819","nonce":1593617983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6823","nonce":1593617983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6821","nonce":1593617983}]},"public_addr":"172.21.15.69:6817/1593617983","cluster_addr":"172.21.15.69:6819/1593617983","heartbeat_back_addr":"172.21.15.69:6823/1593617983","heartbeat_front_addr":"172.21.15.69:6821/1593617983","state":["exists","up"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6809","nonce":3950661294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6811","nonce":3950661294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6815","nonce":3950661294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6813","nonce":3950661294}]},"public_addr":"172.21.15.69:6809/3950661294","cluster_addr":"172.21.15.69:6811/3950661294","heartbeat_back_addr":"172.21.15.69:6815/3950661294","heartbeat_front_addr":"172.21.15.69:6813/3950661294","state":["exists","up"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6801","nonce":2508929872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6803","nonce":2508929872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6807","nonce":2508929872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6805","nonce":2508929872}]},"public_addr":"172.21.15.146:6801/2508929872","cluster_addr":"172.21.15.146:6803/2508929872","heartbeat_back_addr":"172.21.15.146:6807/2508929872","heartbeat_front_addr":"172.21.15.146:6805/2508929872","state":["exists","up"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6811","nonce":1310911138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6816","nonce":1310911138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6823","nonce":1310911138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6821","nonce":1310911138}]},"public_addr":"172.21.15.146:6811/1310911138","cluster_addr":"172.21.15.146:6816/1310911138","heartbeat_back_addr":"172.21.15.146:6823/1310911138","heartbeat_front_addr":"172.21.15.146:6821/1310911138","state":["exists","up"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6810","nonce":634688021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6813","nonce":634688021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6820","nonce":634688021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6817","nonce":634688021}]},"public_addr":"172.21.15.146:6810/634688021","cluster_addr":"172.21.15.146:6813/634688021","heartbeat_back_addr":"172.21.15.146:6820/634688021","heartbeat_front_addr":"172.21.15.146:6817/634688021","state":["exists","up"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6825","nonce":3814215935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6827","nonce":3814215935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6831","nonce":3814215935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6829","nonce":3814215935}]},"public_addr":"172.21.15.146:6825/3814215935","cluster_addr":"172.21.15.146:6827/3814215935","heartbeat_back_addr":"172.21.15.146:6831/3814215935","heartbeat_front_addr":"172.21.15.146:6829/3814215935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.820200+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.850189+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.764560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.866248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.807897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.869881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.782310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.827131+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:59:55.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.687+0000 7f5b99306640 1 -- 172.21.15.69:0/3476688721 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5b740744e0 msgr2=0x7f5b740769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.687+0000 7f5b99306640 1 --2- 172.21.15.69:0/3476688721 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5b740744e0 0x7f5b740769f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5b941fa070 tx=0x7f5b7c009040 comp rx=0 tx=0).stop 2024-04-25T10:59:55.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.687+0000 7f5b99306640 1 -- 172.21.15.69:0/3476688721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b9406c5c0 msgr2=0x7f5b940ccdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.687+0000 7f5b99306640 1 --2- 172.21.15.69:0/3476688721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b9406c5c0 0x7f5b940ccdd0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5b8400e8a0 tx=0x7f5b8400ed60 comp rx=0 tx=0).stop 2024-04-25T10:59:55.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.687+0000 7f5b99306640 1 -- 172.21.15.69:0/3476688721 shutdown_connections 2024-04-25T10:59:55.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.687+0000 7f5b99306640 1 --2- 172.21.15.69:0/3476688721 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5b740744e0 0x7f5b740769f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.687+0000 7f5b99306640 1 --2- 172.21.15.69:0/3476688721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b940d4a50 0x7f5b940c9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.687+0000 7f5b99306640 1 --2- 172.21.15.69:0/3476688721 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5b940d0e00 0x7f5b94154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.687+0000 7f5b99306640 1 --2- 172.21.15.69:0/3476688721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b9406c5c0 0x7f5b940ccdd0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.687+0000 7f5b99306640 1 -- 172.21.15.69:0/3476688721 >> 172.21.15.69:0/3476688721 conn(0x7f5b940d5d90 msgr2=0x7f5b94160070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:55.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.687+0000 7f5b99306640 1 -- 172.21.15.69:0/3476688721 shutdown_connections 2024-04-25T10:59:55.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.687+0000 7f5b99306640 1 -- 172.21.15.69:0/3476688721 wait complete. 2024-04-25T10:59:55.699 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:59:55.700 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:59:55.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.806+0000 7f30f51d5640 1 Processor -- start 2024-04-25T10:59:55.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.807+0000 7f30f51d5640 1 -- start start 2024-04-25T10:59:55.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.808+0000 7f30f51d5640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30f006d8e0 0x7f30f00d3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.808+0000 7f30f51d5640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f30f006dcf0 0x7f30f00d3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.808+0000 7f30f51d5640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30f00b3c10 con 0x7f30f00d04a0 2024-04-25T10:59:55.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.808+0000 7f30f51d5640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f30f00b9520 con 0x7f30f006dcf0 2024-04-25T10:59:55.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.808+0000 7f30f51d5640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f30f00b4820 con 0x7f30f006d8e0 2024-04-25T10:59:55.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.808+0000 7f30ef59e640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f30f00d04a0 0x7f30f00d08b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:34706/0 (socket says 172.21.15.69:34706) 2024-04-25T10:59:55.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.808+0000 7f30ef59e640 1 -- 172.21.15.69:0/2000998463 learned_addr learned my addr 172.21.15.69:0/2000998463 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:55.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.808+0000 7f30e659c640 1 --2- 172.21.15.69:0/2000998463 >> v2:172.21.15.146:3300/0 conn(0x7f30f006dcf0 0x7f30f00d3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.808+0000 7f30eed9d640 1 --2- 172.21.15.69:0/2000998463 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30f006d8e0 0x7f30f00d3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.808+0000 7f30ee59c640 1 -- 172.21.15.69:0/2000998463 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1175768125 0 0) 0x7f30f00b3c10 con 0x7f30f00d04a0 2024-04-25T10:59:55.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.808+0000 7f30ee59c640 1 -- 172.21.15.69:0/2000998463 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f30dc003520 con 0x7f30f00d04a0 2024-04-25T10:59:55.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.808+0000 7f30ee59c640 1 -- 172.21.15.69:0/2000998463 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1240542152 0 0) 0x7f30dc003520 con 0x7f30f00d04a0 2024-04-25T10:59:55.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.808+0000 7f30ee59c640 1 -- 172.21.15.69:0/2000998463 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30f006d8e0 msgr2=0x7f30f00d3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:55.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.808+0000 7f30ee59c640 1 --2- 172.21.15.69:0/2000998463 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30f006d8e0 0x7f30f00d3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.808+0000 7f30ee59c640 1 -- 172.21.15.69:0/2000998463 >> v2:172.21.15.146:3300/0 conn(0x7f30f006dcf0 msgr2=0x7f30f00d3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:55.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.808+0000 7f30ee59c640 1 --2- 172.21.15.69:0/2000998463 >> v2:172.21.15.146:3300/0 conn(0x7f30f006dcf0 0x7f30f00d3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.808+0000 7f30ee59c640 1 -- 172.21.15.69:0/2000998463 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f00b3ef0 con 0x7f30f00d04a0 2024-04-25T10:59:55.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.808+0000 7f30ee59c640 1 -- 172.21.15.69:0/2000998463 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7f30e80019f0 con 0x7f30f00d04a0 2024-04-25T10:59:55.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.809+0000 7f30ee59c640 1 -- 172.21.15.69:0/2000998463 >> v1:172.21.15.69:6789/0 conn(0x7f30f00d04a0 legacy=0x7f30f00d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.809+0000 7f30ee59c640 1 --2- 172.21.15.69:0/2000998463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30dc0039b0 0x7f30dc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.809+0000 7f30ee59c640 1 --2- 172.21.15.69:0/2000998463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30dc004800 0x7f30dc004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.809+0000 7f30ee59c640 1 --2- 172.21.15.69:0/2000998463 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30f006d8e0 0x7f30dc005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.809+0000 7f30ee59c640 1 -- 172.21.15.69:0/2000998463 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f30f00b9520 con 0x7f30dc004800 2024-04-25T10:59:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.809+0000 7f30ee59c640 1 -- 172.21.15.69:0/2000998463 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f30f00b4820 con 0x7f30dc0039b0 2024-04-25T10:59:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.809+0000 7f30ee59c640 1 -- 172.21.15.69:0/2000998463 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f30dc00ba70 con 0x7f30f006d8e0 2024-04-25T10:59:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.809+0000 7f30e659c640 1 --2- 172.21.15.69:0/2000998463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30dc004800 0x7f30dc004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.809+0000 7f30eed9d640 1 --2- 172.21.15.69:0/2000998463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30dc0039b0 0x7f30dc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.809+0000 7f30ef59e640 1 --2- 172.21.15.69:0/2000998463 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30f006d8e0 0x7f30dc005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.809+0000 7f30e659c640 1 -- 172.21.15.69:0/2000998463 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30f006d8e0 msgr2=0x7f30dc005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.809+0000 7f30e659c640 1 --2- 172.21.15.69:0/2000998463 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30f006d8e0 0x7f30dc005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.809+0000 7f30e659c640 1 -- 172.21.15.69:0/2000998463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30dc0039b0 msgr2=0x7f30dc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.809+0000 7f30e659c640 1 --2- 172.21.15.69:0/2000998463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30dc0039b0 0x7f30dc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.809+0000 7f30e659c640 1 -- 172.21.15.69:0/2000998463 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30dc003520 con 0x7f30dc004800 2024-04-25T10:59:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.810+0000 7f30e659c640 1 --2- 172.21.15.69:0/2000998463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30dc004800 0x7f30dc004c10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f30f006f790 tx=0x7f30dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6983714b42f9d527 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.810+0000 7f30ee59c640 1 -- 172.21.15.69:0/2000998463 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d8036070 con 0x7f30dc004800 2024-04-25T10:59:55.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.810+0000 7f30ee59c640 1 -- 172.21.15.69:0/2000998463 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30d8031400 con 0x7f30dc004800 2024-04-25T10:59:55.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.810+0000 7f30ee59c640 1 -- 172.21.15.69:0/2000998463 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d8009690 con 0x7f30dc004800 2024-04-25T10:59:55.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.810+0000 7f30f51d5640 1 -- 172.21.15.69:0/2000998463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30dc004800 msgr2=0x7f30dc004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.810+0000 7f30f51d5640 1 --2- 172.21.15.69:0/2000998463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30dc004800 0x7f30dc004c10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f30f006f790 tx=0x7f30dc005830 comp rx=0 tx=0).stop 2024-04-25T10:59:55.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.810+0000 7f30ef59e640 1 -- 172.21.15.69:0/2000998463 reap_dead start 2024-04-25T10:59:55.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.810+0000 7f30f51d5640 1 -- 172.21.15.69:0/2000998463 shutdown_connections 2024-04-25T10:59:55.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.810+0000 7f30f51d5640 1 -- 172.21.15.69:0/2000998463 >> 172.21.15.69:0/2000998463 conn(0x7f30f008d110 msgr2=0x7f30f008d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:55.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.811+0000 7f30f51d5640 1 -- 172.21.15.69:0/2000998463 shutdown_connections 2024-04-25T10:59:55.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.811+0000 7f30f51d5640 1 -- 172.21.15.69:0/2000998463 wait complete. 2024-04-25T10:59:55.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.812+0000 7f30f51d5640 1 Processor -- start 2024-04-25T10:59:55.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.813+0000 7f30f51d5640 1 -- start start 2024-04-25T10:59:55.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.814+0000 7f30f51d5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30dc004800 0x7f30f0153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.814+0000 7f30f51d5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30f01e6a30 0x7f30f01e6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.814+0000 7f30f51d5640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30f00a0bb0 0x7f30f00a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.814+0000 7f30f51d5640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f30f00b38e0 con 0x7f30dc004800 2024-04-25T10:59:55.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.814+0000 7f30f51d5640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f30f00b44b0 con 0x7f30f01e6a30 2024-04-25T10:59:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.814+0000 7f30f51d5640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f30f00d0fc0 con 0x7f30f00a0bb0 2024-04-25T10:59:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.814+0000 7f30eed9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30dc004800 0x7f30f0153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.814+0000 7f30eed9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30dc004800 0x7f30f0153f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47998/0 (socket says 172.21.15.69:47998) 2024-04-25T10:59:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.814+0000 7f30eed9d640 1 -- 172.21.15.69:0/2319422591 learned_addr learned my addr 172.21.15.69:0/2319422591 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.814+0000 7f30eed9d640 1 -- 172.21.15.69:0/2319422591 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30f00a0bb0 msgr2=0x7f30f00a0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.814+0000 7f30eed9d640 1 --2- 172.21.15.69:0/2319422591 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30f00a0bb0 0x7f30f00a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.814+0000 7f30e659c640 1 --2- 172.21.15.69:0/2319422591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30f01e6a30 0x7f30f01e6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.814+0000 7f30eed9d640 1 -- 172.21.15.69:0/2319422591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30f01e6a30 msgr2=0x7f30f01e6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.814+0000 7f30eed9d640 1 --2- 172.21.15.69:0/2319422591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30f01e6a30 0x7f30f01e6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.814+0000 7f30eed9d640 1 -- 172.21.15.69:0/2319422591 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f00a1500 con 0x7f30dc004800 2024-04-25T10:59:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.815+0000 7f30eed9d640 1 --2- 172.21.15.69:0/2319422591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30dc004800 0x7f30f0153f10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f30e00118e0 tx=0x7f30e0011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.815+0000 7f30e7fff640 1 -- 172.21.15.69:0/2319422591 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e000fa30 con 0x7f30dc004800 2024-04-25T10:59:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.815+0000 7f30e7fff640 1 -- 172.21.15.69:0/2319422591 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30e000fbd0 con 0x7f30dc004800 2024-04-25T10:59:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.815+0000 7f30e7fff640 1 -- 172.21.15.69:0/2319422591 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e0014440 con 0x7f30dc004800 2024-04-25T10:59:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.815+0000 7f30f51d5640 1 -- 172.21.15.69:0/2319422591 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30f01e76d0 con 0x7f30dc004800 2024-04-25T10:59:55.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.815+0000 7f30f51d5640 1 -- 172.21.15.69:0/2319422591 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30f01e7c90 con 0x7f30dc004800 2024-04-25T10:59:55.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.816+0000 7f30f51d5640 1 -- 172.21.15.69:0/2319422591 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30f0050740 con 0x7f30dc004800 2024-04-25T10:59:55.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.816+0000 7f30e7fff640 1 -- 172.21.15.69:0/2319422591 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f30e0012040 con 0x7f30dc004800 2024-04-25T10:59:55.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.817+0000 7f30e7fff640 1 --2- 172.21.15.69:0/2319422591 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f30d00744e0 0x7f30d00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:55.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.817+0000 7f30e7fff640 1 -- 172.21.15.69:0/2319422591 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f30e0095710 con 0x7f30dc004800 2024-04-25T10:59:55.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.817+0000 7f30e659c640 1 --2- 172.21.15.69:0/2319422591 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f30d00744e0 0x7f30d00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:55.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.817+0000 7f30e659c640 1 --2- 172.21.15.69:0/2319422591 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f30d00744e0 0x7f30d00769f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f30f006f790 tx=0x7f30d8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:55.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:55.822+0000 7f30e7fff640 1 -- 172.21.15.69:0/2319422591 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f30f00a1500 con 0x7f30dc004800 2024-04-25T10:59:56.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.031+0000 7f30f51d5640 1 -- 172.21.15.69:0/2319422591 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30f00d40a0 con 0x7f30dc004800 2024-04-25T10:59:56.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.032+0000 7f30e7fff640 1 -- 172.21.15.69:0/2319422591 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5178 (secure 0 0 0) 0x7f30e0019030 con 0x7f30dc004800 2024-04-25T10:59:56.033 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:56.033 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/2361834023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2361834023},{"type":"v1","addr":"172.21.15.146:6833","nonce":2361834023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1848276963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1848276963},{"type":"v1","addr":"172.21.15.69:6837","nonce":1848276963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1826742846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1826742846},{"type":"v1","addr":"172.21.15.69:6836","nonce":1826742846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:59:18.976232+0000","modified":"2024-04-25T10:59:20.983284+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6835/908008797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":908008797},{"type":"v1","addr":"172.21.15.146:6835","nonce":908008797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-04-25T10:59:56.033 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 6 2024-04-25T10:59:56.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.034+0000 7f30f51d5640 1 -- 172.21.15.69:0/2319422591 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f30d00744e0 msgr2=0x7f30d00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:56.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.034+0000 7f30f51d5640 1 --2- 172.21.15.69:0/2319422591 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f30d00744e0 0x7f30d00769f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f30f006f790 tx=0x7f30d8008030 comp rx=0 tx=0).stop 2024-04-25T10:59:56.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.034+0000 7f30f51d5640 1 -- 172.21.15.69:0/2319422591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30dc004800 msgr2=0x7f30f0153f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:56.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.034+0000 7f30f51d5640 1 --2- 172.21.15.69:0/2319422591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30dc004800 0x7f30f0153f10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f30e00118e0 tx=0x7f30e0011da0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.034+0000 7f30f51d5640 1 -- 172.21.15.69:0/2319422591 shutdown_connections 2024-04-25T10:59:56.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.034+0000 7f30f51d5640 1 --2- 172.21.15.69:0/2319422591 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f30d00744e0 0x7f30d00769f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.034+0000 7f30f51d5640 1 --2- 172.21.15.69:0/2319422591 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30f00a0bb0 0x7f30f00a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.034+0000 7f30f51d5640 1 --2- 172.21.15.69:0/2319422591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30f01e6a30 0x7f30f01e6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.034+0000 7f30f51d5640 1 --2- 172.21.15.69:0/2319422591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30dc004800 0x7f30f0153f10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.034+0000 7f30f51d5640 1 -- 172.21.15.69:0/2319422591 >> 172.21.15.69:0/2319422591 conn(0x7f30f008d110 msgr2=0x7f30f00b8660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:56.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.034+0000 7f30f51d5640 1 -- 172.21.15.69:0/2319422591 shutdown_connections 2024-04-25T10:59:56.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.034+0000 7f30f51d5640 1 -- 172.21.15.69:0/2319422591 wait complete. 2024-04-25T10:59:56.046 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:59:56.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.151+0000 7fa0729b2640 1 Processor -- start 2024-04-25T10:59:56.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.152+0000 7fa0729b2640 1 -- start start 2024-04-25T10:59:56.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.153+0000 7fa0729b2640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fa06c0b21a0 0x7fa06c0ce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:56.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.153+0000 7fa0729b2640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa06c0d1c10 con 0x7fa06c0b7840 2024-04-25T10:59:56.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.153+0000 7fa0729b2640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa06c0b25b0 con 0x7fa06c0cfd40 2024-04-25T10:59:56.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.153+0000 7fa0729b2640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fa06c0d27e0 con 0x7fa06c0b21a0 2024-04-25T10:59:56.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.153+0000 7fa06b7fe640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fa06c0b21a0 0x7fa06c0ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:56.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.153+0000 7fa070f4f640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fa06c0b7840 0x7fa06c0b7c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:34718/0 (socket says 172.21.15.69:34718) 2024-04-25T10:59:56.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.153+0000 7fa070f4f640 1 -- 172.21.15.69:0/3650970842 learned_addr learned my addr 172.21.15.69:0/3650970842 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:56.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.153+0000 7fa06b7fe640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fa06c0b21a0 0x7fa06c0ce110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48002/0 (socket says 172.21.15.69:48002) 2024-04-25T10:59:56.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.153+0000 7fa06b7fe640 1 -- 172.21.15.69:0/3650970842 >> v1:172.21.15.146:6789/0 conn(0x7fa06c0cfd40 legacy=0x7fa06c0b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:56.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.153+0000 7fa06b7fe640 1 -- 172.21.15.69:0/3650970842 >> v1:172.21.15.69:6789/0 conn(0x7fa06c0b7840 legacy=0x7fa06c0b7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:56.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.153+0000 7fa06b7fe640 1 -- 172.21.15.69:0/3650970842 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06c0d1eb0 con 0x7fa06c0b21a0 2024-04-25T10:59:56.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.154+0000 7fa06b7fe640 1 --2- 172.21.15.69:0/3650970842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa06c0b21a0 0x7fa06c0ce110 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa0600097a0 tx=0x7fa06002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856b5a68004dae76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:56.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.154+0000 7fa06affd640 1 -- 172.21.15.69:0/3650970842 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa060037070 con 0x7fa06c0b21a0 2024-04-25T10:59:56.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.154+0000 7fa06affd640 1 -- 172.21.15.69:0/3650970842 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa06002ae50 con 0x7fa06c0b21a0 2024-04-25T10:59:56.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.154+0000 7fa06affd640 1 -- 172.21.15.69:0/3650970842 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa060032660 con 0x7fa06c0b21a0 2024-04-25T10:59:56.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.155+0000 7fa0729b2640 1 -- 172.21.15.69:0/3650970842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa06c0b21a0 msgr2=0x7fa06c0ce110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:56.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.155+0000 7fa0729b2640 1 --2- 172.21.15.69:0/3650970842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa06c0b21a0 0x7fa06c0ce110 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa0600097a0 tx=0x7fa06002a140 comp rx=0 tx=0).stop 2024-04-25T10:59:56.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.155+0000 7fa0729b2640 1 -- 172.21.15.69:0/3650970842 shutdown_connections 2024-04-25T10:59:56.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.155+0000 7fa0729b2640 1 --2- 172.21.15.69:0/3650970842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa06c0b21a0 0x7fa06c0ce110 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.155+0000 7fa0729b2640 1 -- 172.21.15.69:0/3650970842 >> 172.21.15.69:0/3650970842 conn(0x7fa06c09b7d0 msgr2=0x7fa06c09bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:56.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.155+0000 7fa0729b2640 1 -- 172.21.15.69:0/3650970842 shutdown_connections 2024-04-25T10:59:56.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.155+0000 7fa0729b2640 1 -- 172.21.15.69:0/3650970842 wait complete. 2024-04-25T10:59:56.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.157+0000 7fa0729b2640 1 Processor -- start 2024-04-25T10:59:56.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.157+0000 7fa0729b2640 1 -- start start 2024-04-25T10:59:56.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.158+0000 7fa0729b2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa06c0b21a0 0x7fa06c151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:56.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.158+0000 7fa0729b2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa06c0b7840 0x7fa06c0ca570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:56.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.158+0000 7fa0729b2640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa06c0cfd40 0x7fa06c0cd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:56.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.158+0000 7fa0729b2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c0ce650 con 0x7fa06c0b21a0 2024-04-25T10:59:56.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.158+0000 7fa0729b2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c0d2470 con 0x7fa06c0b7840 2024-04-25T10:59:56.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.158+0000 7fa0729b2640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa06c0d2c70 con 0x7fa06c0cfd40 2024-04-25T10:59:56.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.158+0000 7fa06bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa06c0b21a0 0x7fa06c151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:56.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.158+0000 7fa06bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa06c0b21a0 0x7fa06c151c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48014/0 (socket says 172.21.15.69:48014) 2024-04-25T10:59:56.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.158+0000 7fa06bfff640 1 -- 172.21.15.69:0/2945618420 learned_addr learned my addr 172.21.15.69:0/2945618420 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:56.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.158+0000 7fa070f4f640 1 --2- 172.21.15.69:0/2945618420 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa06c0cfd40 0x7fa06c0cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:56.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.159+0000 7fa06b7fe640 1 --2- 172.21.15.69:0/2945618420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa06c0b7840 0x7fa06c0ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:56.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.159+0000 7fa06bfff640 1 -- 172.21.15.69:0/2945618420 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa06c0cfd40 msgr2=0x7fa06c0cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:56.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.159+0000 7fa06bfff640 1 --2- 172.21.15.69:0/2945618420 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa06c0cfd40 0x7fa06c0cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.159+0000 7fa06bfff640 1 -- 172.21.15.69:0/2945618420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa06c0b7840 msgr2=0x7fa06c0ca570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:56.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.159+0000 7fa06bfff640 1 --2- 172.21.15.69:0/2945618420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa06c0b7840 0x7fa06c0ca570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.159+0000 7fa06bfff640 1 -- 172.21.15.69:0/2945618420 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06c0cb890 con 0x7fa06c0b21a0 2024-04-25T10:59:56.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.159+0000 7fa070f4f640 1 --2- 172.21.15.69:0/2945618420 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa06c0cfd40 0x7fa06c0cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:56.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.159+0000 7fa06b7fe640 1 --2- 172.21.15.69:0/2945618420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa06c0b7840 0x7fa06c0ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:56.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.159+0000 7fa06bfff640 1 --2- 172.21.15.69:0/2945618420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa06c0b21a0 0x7fa06c151c80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00b420 tx=0x7fa05c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:56.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.159+0000 7fa068ff9640 1 -- 172.21.15.69:0/2945618420 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c011830 con 0x7fa06c0b21a0 2024-04-25T10:59:56.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.159+0000 7fa068ff9640 1 -- 172.21.15.69:0/2945618420 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa05c016070 con 0x7fa06c0b21a0 2024-04-25T10:59:56.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.159+0000 7fa0729b2640 1 -- 172.21.15.69:0/2945618420 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa06c0cbb70 con 0x7fa06c0b21a0 2024-04-25T10:59:56.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.159+0000 7fa068ff9640 1 -- 172.21.15.69:0/2945618420 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c011c10 con 0x7fa06c0b21a0 2024-04-25T10:59:56.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.159+0000 7fa0729b2640 1 -- 172.21.15.69:0/2945618420 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa06c0cc130 con 0x7fa06c0b21a0 2024-04-25T10:59:56.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.160+0000 7fa0729b2640 1 -- 172.21.15.69:0/2945618420 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa06c050740 con 0x7fa06c0b21a0 2024-04-25T10:59:56.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.161+0000 7fa068ff9640 1 -- 172.21.15.69:0/2945618420 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa05c01b460 con 0x7fa06c0b21a0 2024-04-25T10:59:56.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.161+0000 7fa068ff9640 1 --2- 172.21.15.69:0/2945618420 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa0580744e0 0x7fa0580769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:56.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.161+0000 7fa06b7fe640 1 --2- 172.21.15.69:0/2945618420 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa0580744e0 0x7fa0580769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:56.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.161+0000 7fa068ff9640 1 -- 172.21.15.69:0/2945618420 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa05c094d30 con 0x7fa06c0b21a0 2024-04-25T10:59:56.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.162+0000 7fa06b7fe640 1 --2- 172.21.15.69:0/2945618420 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa0580744e0 0x7fa0580769f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa060034000 tx=0x7fa06002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:56.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.166+0000 7fa068ff9640 1 -- 172.21.15.69:0/2945618420 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa05c020020 con 0x7fa06c0b21a0 2024-04-25T10:59:56.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.376+0000 7fa0729b2640 1 -- 172.21.15.69:0/2945618420 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa06c0ccb20 con 0x7fa06c0b21a0 2024-04-25T10:59:56.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.377+0000 7fa068ff9640 1 -- 172.21.15.69:0/2945618420 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5178 (secure 0 0 0) 0x7fa05c05f4d0 con 0x7fa06c0b21a0 2024-04-25T10:59:56.377 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:56.378 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/2361834023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2361834023},{"type":"v1","addr":"172.21.15.146:6833","nonce":2361834023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1848276963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1848276963},{"type":"v1","addr":"172.21.15.69:6837","nonce":1848276963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/1826742846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":1826742846},{"type":"v1","addr":"172.21.15.69:6836","nonce":1826742846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:59:18.976232+0000","modified":"2024-04-25T10:59:20.983284+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6835/908008797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":908008797},{"type":"v1","addr":"172.21.15.146:6835","nonce":908008797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-04-25T10:59:56.378 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 6 2024-04-25T10:59:56.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.379+0000 7fa0729b2640 1 -- 172.21.15.69:0/2945618420 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa0580744e0 msgr2=0x7fa0580769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:56.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.379+0000 7fa0729b2640 1 --2- 172.21.15.69:0/2945618420 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa0580744e0 0x7fa0580769f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa060034000 tx=0x7fa06002e000 comp rx=0 tx=0).stop 2024-04-25T10:59:56.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.379+0000 7fa0729b2640 1 -- 172.21.15.69:0/2945618420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa06c0b21a0 msgr2=0x7fa06c151c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:56.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.379+0000 7fa0729b2640 1 --2- 172.21.15.69:0/2945618420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa06c0b21a0 0x7fa06c151c80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00b420 tx=0x7fa05c00b8e0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.379+0000 7fa0729b2640 1 -- 172.21.15.69:0/2945618420 shutdown_connections 2024-04-25T10:59:56.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.379+0000 7fa0729b2640 1 --2- 172.21.15.69:0/2945618420 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa0580744e0 0x7fa0580769f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.379+0000 7fa0729b2640 1 --2- 172.21.15.69:0/2945618420 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa06c0cfd40 0x7fa06c0cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.379+0000 7fa0729b2640 1 --2- 172.21.15.69:0/2945618420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa06c0b7840 0x7fa06c0ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.379+0000 7fa0729b2640 1 --2- 172.21.15.69:0/2945618420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa06c0b21a0 0x7fa06c151c80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.379+0000 7fa0729b2640 1 -- 172.21.15.69:0/2945618420 >> 172.21.15.69:0/2945618420 conn(0x7fa06c09b7d0 msgr2=0x7fa06c0ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:56.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.379+0000 7fa0729b2640 1 -- 172.21.15.69:0/2945618420 shutdown_connections 2024-04-25T10:59:56.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.379+0000 7fa0729b2640 1 -- 172.21.15.69:0/2945618420 wait complete. 2024-04-25T10:59:56.391 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:59:56.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.497+0000 7ff61f86e640 1 Processor -- start 2024-04-25T10:59:56.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.497+0000 7ff61f86e640 1 -- start start 2024-04-25T10:59:56.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.499+0000 7ff61f86e640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff6180d33d0 0x7ff6180b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:56.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.499+0000 7ff61f86e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6180d0be0 0x7ff6180ce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:56.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.499+0000 7ff61f86e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6180d2330 con 0x7ff6180d45e0 2024-04-25T10:59:56.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.499+0000 7ff61f86e640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7ff6180d2610 con 0x7ff6180d33d0 2024-04-25T10:59:56.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.499+0000 7ff61f86e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff6180d28f0 con 0x7ff6180d0be0 2024-04-25T10:59:56.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.499+0000 7ff61d60a640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff6180d33d0 0x7ff6180b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:56.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.499+0000 7ff61d60a640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff6180d33d0 0x7ff6180b4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48028/0 (socket says 172.21.15.69:48028) 2024-04-25T10:59:56.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.499+0000 7ff61d60a640 1 -- 172.21.15.69:0/2353006486 learned_addr learned my addr 172.21.15.69:0/2353006486 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:56.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.499+0000 7ff61de0b640 1 --2- 172.21.15.69:0/2353006486 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6180d0be0 0x7ff6180ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:56.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.499+0000 7ff61d60a640 1 -- 172.21.15.69:0/2353006486 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6180d0be0 msgr2=0x7ff6180ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:56.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.499+0000 7ff61d60a640 1 --2- 172.21.15.69:0/2353006486 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6180d0be0 0x7ff6180ce110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.499+0000 7ff61d60a640 1 -- 172.21.15.69:0/2353006486 >> v1:172.21.15.146:6789/0 conn(0x7ff6180d45e0 legacy=0x7ff6180d04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:56.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.499+0000 7ff61d60a640 1 -- 172.21.15.69:0/2353006486 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6180d2bd0 con 0x7ff6180d33d0 2024-04-25T10:59:56.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.500+0000 7ff61d60a640 1 --2- 172.21.15.69:0/2353006486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6180d33d0 0x7ff6180b4fb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff614009850 tx=0x7ff61402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23f0c8b69a50522 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:56.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.500+0000 7ff60ffff640 1 -- 172.21.15.69:0/2353006486 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff614004030 con 0x7ff6180d33d0 2024-04-25T10:59:56.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.500+0000 7ff60ffff640 1 -- 172.21.15.69:0/2353006486 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6140041d0 con 0x7ff6180d33d0 2024-04-25T10:59:56.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.500+0000 7ff60ffff640 1 -- 172.21.15.69:0/2353006486 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff614003610 con 0x7ff6180d33d0 2024-04-25T10:59:56.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.501+0000 7ff61f86e640 1 -- 172.21.15.69:0/2353006486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6180d33d0 msgr2=0x7ff6180b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:56.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.501+0000 7ff61f86e640 1 --2- 172.21.15.69:0/2353006486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6180d33d0 0x7ff6180b4fb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff614009850 tx=0x7ff61402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.501+0000 7ff61f86e640 1 -- 172.21.15.69:0/2353006486 shutdown_connections 2024-04-25T10:59:56.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.501+0000 7ff61f86e640 1 --2- 172.21.15.69:0/2353006486 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6180d0be0 0x7ff6180ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.501+0000 7ff61f86e640 1 --2- 172.21.15.69:0/2353006486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6180d33d0 0x7ff6180b4fb0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.501+0000 7ff61f86e640 1 -- 172.21.15.69:0/2353006486 >> 172.21.15.69:0/2353006486 conn(0x7ff6180d3ac0 msgr2=0x7ff6180d3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:56.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.501+0000 7ff61f86e640 1 -- 172.21.15.69:0/2353006486 shutdown_connections 2024-04-25T10:59:56.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.501+0000 7ff61f86e640 1 -- 172.21.15.69:0/2353006486 wait complete. 2024-04-25T10:59:56.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.502+0000 7ff61f86e640 1 Processor -- start 2024-04-25T10:59:56.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.503+0000 7ff61f86e640 1 -- start start 2024-04-25T10:59:56.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.504+0000 7ff61f86e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6180d33d0 0x7ff61814ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:56.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.504+0000 7ff61f86e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6180d45e0 0x7ff61814d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:56.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.504+0000 7ff61f86e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6180b07b0 0x7ff61814d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:56.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.504+0000 7ff61f86e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff6180d2040 con 0x7ff6180b07b0 2024-04-25T10:59:56.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.504+0000 7ff61f86e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff6180d1ec0 con 0x7ff6180d33d0 2024-04-25T10:59:56.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.504+0000 7ff61f86e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff61806dc70 con 0x7ff6180d45e0 2024-04-25T10:59:56.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.504+0000 7ff61de0b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6180b07b0 0x7ff61814d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:56.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.504+0000 7ff61de0b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6180b07b0 0x7ff61814d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48030/0 (socket says 172.21.15.69:48030) 2024-04-25T10:59:56.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.504+0000 7ff61de0b640 1 -- 172.21.15.69:0/960786360 learned_addr learned my addr 172.21.15.69:0/960786360 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:56.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.504+0000 7ff61d60a640 1 --2- 172.21.15.69:0/960786360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6180d33d0 0x7ff61814ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:56.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.504+0000 7ff61ce09640 1 --2- 172.21.15.69:0/960786360 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6180d45e0 0x7ff61814d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:56.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.504+0000 7ff61de0b640 1 -- 172.21.15.69:0/960786360 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6180d45e0 msgr2=0x7ff61814d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:56.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.504+0000 7ff61de0b640 1 --2- 172.21.15.69:0/960786360 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6180d45e0 0x7ff61814d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.504+0000 7ff61de0b640 1 -- 172.21.15.69:0/960786360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6180d33d0 msgr2=0x7ff61814ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:56.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.504+0000 7ff61de0b640 1 --2- 172.21.15.69:0/960786360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6180d33d0 0x7ff61814ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.504+0000 7ff61de0b640 1 -- 172.21.15.69:0/960786360 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff61814c400 con 0x7ff6180b07b0 2024-04-25T10:59:56.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.504+0000 7ff61de0b640 1 --2- 172.21.15.69:0/960786360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6180b07b0 0x7ff61814d700 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff610007d40 tx=0x7ff61000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:56.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.504+0000 7ff60dffb640 1 -- 172.21.15.69:0/960786360 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff610014420 con 0x7ff6180b07b0 2024-04-25T10:59:56.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.505+0000 7ff60dffb640 1 -- 172.21.15.69:0/960786360 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff610019070 con 0x7ff6180b07b0 2024-04-25T10:59:56.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.505+0000 7ff61f86e640 1 -- 172.21.15.69:0/960786360 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff61814c6e0 con 0x7ff6180b07b0 2024-04-25T10:59:56.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.505+0000 7ff60dffb640 1 -- 172.21.15.69:0/960786360 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff610014820 con 0x7ff6180b07b0 2024-04-25T10:59:56.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.505+0000 7ff61f86e640 1 -- 172.21.15.69:0/960786360 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff61814cca0 con 0x7ff6180b07b0 2024-04-25T10:59:56.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.506+0000 7ff61f86e640 1 -- 172.21.15.69:0/960786360 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff618050740 con 0x7ff6180b07b0 2024-04-25T10:59:56.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.506+0000 7ff60dffb640 1 -- 172.21.15.69:0/960786360 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff6100149c0 con 0x7ff6180b07b0 2024-04-25T10:59:56.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.506+0000 7ff60dffb640 1 --2- 172.21.15.69:0/960786360 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff5fc0744b0 0x7ff5fc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:56.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.507+0000 7ff61d60a640 1 --2- 172.21.15.69:0/960786360 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff5fc0744b0 0x7ff5fc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:56.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.507+0000 7ff60dffb640 1 -- 172.21.15.69:0/960786360 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff610095a30 con 0x7ff6180b07b0 2024-04-25T10:59:56.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.507+0000 7ff61d60a640 1 --2- 172.21.15.69:0/960786360 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff5fc0744b0 0x7ff5fc0769c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff614033000 tx=0x7ff61402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:56.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.511+0000 7ff60dffb640 1 -- 172.21.15.69:0/960786360 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff6100601c0 con 0x7ff6180b07b0 2024-04-25T10:59:56.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.686+0000 7ff61f86e640 1 -- 172.21.15.69:0/960786360 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff6180d3f70 con 0x7ff6180b07b0 2024-04-25T10:59:56.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.688+0000 7ff60dffb640 1 -- 172.21.15.69:0/960786360 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7ff61005fac0 con 0x7ff6180b07b0 2024-04-25T10:59:56.688 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T10:59:56.688 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":20,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T10:59:20.630151+0000","last_up_change":"2024-04-25T10:59:05.546839+0000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:59:06.422674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:59:16.727980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:59:17.938486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6825","nonce":1876101065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6827","nonce":1876101065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6831","nonce":1876101065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6829","nonce":1876101065}]},"public_addr":"172.21.15.69:6825/1876101065","cluster_addr":"172.21.15.69:6827/1876101065","heartbeat_back_addr":"172.21.15.69:6831/1876101065","heartbeat_front_addr":"172.21.15.69:6829/1876101065","state":["exists","up"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6801","nonce":1806438563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6803","nonce":1806438563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6807","nonce":1806438563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6805","nonce":1806438563}]},"public_addr":"172.21.15.69:6801/1806438563","cluster_addr":"172.21.15.69:6803/1806438563","heartbeat_back_addr":"172.21.15.69:6807/1806438563","heartbeat_front_addr":"172.21.15.69:6805/1806438563","state":["exists","up"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6817","nonce":1593617983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6819","nonce":1593617983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6823","nonce":1593617983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6821","nonce":1593617983}]},"public_addr":"172.21.15.69:6817/1593617983","cluster_addr":"172.21.15.69:6819/1593617983","heartbeat_back_addr":"172.21.15.69:6823/1593617983","heartbeat_front_addr":"172.21.15.69:6821/1593617983","state":["exists","up"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6809","nonce":3950661294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6811","nonce":3950661294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6815","nonce":3950661294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6813","nonce":3950661294}]},"public_addr":"172.21.15.69:6809/3950661294","cluster_addr":"172.21.15.69:6811/3950661294","heartbeat_back_addr":"172.21.15.69:6815/3950661294","heartbeat_front_addr":"172.21.15.69:6813/3950661294","state":["exists","up"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6801","nonce":2508929872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6803","nonce":2508929872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6807","nonce":2508929872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6805","nonce":2508929872}]},"public_addr":"172.21.15.146:6801/2508929872","cluster_addr":"172.21.15.146:6803/2508929872","heartbeat_back_addr":"172.21.15.146:6807/2508929872","heartbeat_front_addr":"172.21.15.146:6805/2508929872","state":["exists","up"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6811","nonce":1310911138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6816","nonce":1310911138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6823","nonce":1310911138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6821","nonce":1310911138}]},"public_addr":"172.21.15.146:6811/1310911138","cluster_addr":"172.21.15.146:6816/1310911138","heartbeat_back_addr":"172.21.15.146:6823/1310911138","heartbeat_front_addr":"172.21.15.146:6821/1310911138","state":["exists","up"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6810","nonce":634688021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6813","nonce":634688021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6820","nonce":634688021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6817","nonce":634688021}]},"public_addr":"172.21.15.146:6810/634688021","cluster_addr":"172.21.15.146:6813/634688021","heartbeat_back_addr":"172.21.15.146:6820/634688021","heartbeat_front_addr":"172.21.15.146:6817/634688021","state":["exists","up"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6825","nonce":3814215935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6827","nonce":3814215935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6831","nonce":3814215935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6829","nonce":3814215935}]},"public_addr":"172.21.15.146:6825/3814215935","cluster_addr":"172.21.15.146:6827/3814215935","heartbeat_back_addr":"172.21.15.146:6831/3814215935","heartbeat_front_addr":"172.21.15.146:6829/3814215935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.820200+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.850189+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.764560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.866248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.807897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.869881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.782310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.827131+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:59:56.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.690+0000 7ff61f86e640 1 -- 172.21.15.69:0/960786360 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff5fc0744b0 msgr2=0x7ff5fc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:56.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.690+0000 7ff61f86e640 1 --2- 172.21.15.69:0/960786360 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff5fc0744b0 0x7ff5fc0769c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff614033000 tx=0x7ff61402e000 comp rx=0 tx=0).stop 2024-04-25T10:59:56.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.690+0000 7ff61f86e640 1 -- 172.21.15.69:0/960786360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6180b07b0 msgr2=0x7ff61814d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:56.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.690+0000 7ff61f86e640 1 --2- 172.21.15.69:0/960786360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6180b07b0 0x7ff61814d700 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff610007d40 tx=0x7ff61000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.690+0000 7ff61f86e640 1 -- 172.21.15.69:0/960786360 shutdown_connections 2024-04-25T10:59:56.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.690+0000 7ff61f86e640 1 --2- 172.21.15.69:0/960786360 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff5fc0744b0 0x7ff5fc0769c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.690+0000 7ff61f86e640 1 --2- 172.21.15.69:0/960786360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6180b07b0 0x7ff61814d700 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.690+0000 7ff61f86e640 1 --2- 172.21.15.69:0/960786360 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6180d45e0 0x7ff61814d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.690+0000 7ff61f86e640 1 --2- 172.21.15.69:0/960786360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6180d33d0 0x7ff61814ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.690+0000 7ff61f86e640 1 -- 172.21.15.69:0/960786360 >> 172.21.15.69:0/960786360 conn(0x7ff6180d3ac0 msgr2=0x7ff618072b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:56.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.690+0000 7ff61f86e640 1 -- 172.21.15.69:0/960786360 shutdown_connections 2024-04-25T10:59:56.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.690+0000 7ff61f86e640 1 -- 172.21.15.69:0/960786360 wait complete. 2024-04-25T10:59:56.702 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:59:56.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.810+0000 7f91d2734640 1 Processor -- start 2024-04-25T10:59:56.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.810+0000 7f91d2734640 1 -- start start 2024-04-25T10:59:56.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.811+0000 7f91d2734640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f91cc0ce800 0x7f91cc0d1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:56.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.811+0000 7f91d2734640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91cc163130 0x7f91cc0b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:56.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.811+0000 7f91d2734640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91cc0b0770 con 0x7f91cc0d1620 2024-04-25T10:59:56.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.811+0000 7f91d2734640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f91cc0b10a0 con 0x7f91cc0ce800 2024-04-25T10:59:56.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.812+0000 7f91d2734640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f91cc0d2100 con 0x7f91cc163130 2024-04-25T10:59:56.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.812+0000 7f91cb7fe640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f91cc0d1620 0x7f91cc162a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:34734/0 (socket says 172.21.15.69:34734) 2024-04-25T10:59:56.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.812+0000 7f91cb7fe640 1 -- 172.21.15.69:0/3192109629 learned_addr learned my addr 172.21.15.69:0/3192109629 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:56.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.812+0000 7f91d0cd1640 1 --2- 172.21.15.69:0/3192109629 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91cc163130 0x7f91cc0b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:56.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.812+0000 7f91cbfff640 1 --2- 172.21.15.69:0/3192109629 >> v2:172.21.15.146:3300/0 conn(0x7f91cc0ce800 0x7f91cc0d1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:56.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.812+0000 7f91caffd640 1 -- 172.21.15.69:0/3192109629 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2420585895 0 0) 0x7f91cc0b0770 con 0x7f91cc0d1620 2024-04-25T10:59:56.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.812+0000 7f91caffd640 1 -- 172.21.15.69:0/3192109629 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91b4003520 con 0x7f91cc0d1620 2024-04-25T10:59:56.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.812+0000 7f91caffd640 1 -- 172.21.15.69:0/3192109629 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2715317741 0 0) 0x7f91b4003520 con 0x7f91cc0d1620 2024-04-25T10:59:56.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.812+0000 7f91caffd640 1 -- 172.21.15.69:0/3192109629 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91cc163130 msgr2=0x7f91cc0b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:56.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.812+0000 7f91caffd640 1 --2- 172.21.15.69:0/3192109629 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91cc163130 0x7f91cc0b2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.812+0000 7f91caffd640 1 -- 172.21.15.69:0/3192109629 >> v2:172.21.15.146:3300/0 conn(0x7f91cc0ce800 msgr2=0x7f91cc0d1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:56.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.812+0000 7f91caffd640 1 --2- 172.21.15.69:0/3192109629 >> v2:172.21.15.146:3300/0 conn(0x7f91cc0ce800 0x7f91cc0d1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.812+0000 7f91caffd640 1 -- 172.21.15.69:0/3192109629 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91cc0b0a50 con 0x7f91cc0d1620 2024-04-25T10:59:56.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.812+0000 7f91caffd640 1 -- 172.21.15.69:0/3192109629 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7f91c0001670 con 0x7f91cc0d1620 2024-04-25T10:59:56.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.812+0000 7f91d0cd1640 1 --2- 172.21.15.69:0/3192109629 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91cc163130 0x7f91cc0b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:59:56.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.812+0000 7f91caffd640 1 -- 172.21.15.69:0/3192109629 >> v1:172.21.15.69:6789/0 conn(0x7f91cc0d1620 legacy=0x7f91cc162a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:56.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.812+0000 7f91caffd640 1 --2- 172.21.15.69:0/3192109629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91b40039b0 0x7f91b4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:56.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.812+0000 7f91caffd640 1 --2- 172.21.15.69:0/3192109629 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91cc163130 0x7f91b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:56.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.812+0000 7f91caffd640 1 --2- 172.21.15.69:0/3192109629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91b4004d40 0x7f91b4005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:56.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.812+0000 7f91caffd640 1 -- 172.21.15.69:0/3192109629 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f91cc0b10a0 con 0x7f91b4004d40 2024-04-25T10:59:56.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.812+0000 7f91caffd640 1 -- 172.21.15.69:0/3192109629 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f91cc0d2100 con 0x7f91b40039b0 2024-04-25T10:59:56.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.812+0000 7f91caffd640 1 -- 172.21.15.69:0/3192109629 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f91b400baa0 con 0x7f91cc163130 2024-04-25T10:59:56.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.812+0000 7f91cbfff640 1 --2- 172.21.15.69:0/3192109629 >> v2:172.21.15.146:3300/0 conn(0x7f91cc0ce800 0x7f91cc0d1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:59:56.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.812+0000 7f91d0cd1640 1 --2- 172.21.15.69:0/3192109629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91b4004d40 0x7f91b4005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:56.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.813+0000 7f91cb7fe640 1 --2- 172.21.15.69:0/3192109629 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91cc163130 0x7f91b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:56.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.813+0000 7f91cbfff640 1 --2- 172.21.15.69:0/3192109629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91b40039b0 0x7f91b4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:56.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.813+0000 7f91d0cd1640 1 -- 172.21.15.69:0/3192109629 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91cc163130 msgr2=0x7f91b4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:56.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.813+0000 7f91d0cd1640 1 --2- 172.21.15.69:0/3192109629 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91cc163130 0x7f91b4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.813+0000 7f91d0cd1640 1 -- 172.21.15.69:0/3192109629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91b40039b0 msgr2=0x7f91b4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:56.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.813+0000 7f91d0cd1640 1 --2- 172.21.15.69:0/3192109629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91b40039b0 0x7f91b4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.813+0000 7f91d0cd1640 1 -- 172.21.15.69:0/3192109629 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b4003520 con 0x7f91b4004d40 2024-04-25T10:59:56.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.813+0000 7f91d0cd1640 1 --2- 172.21.15.69:0/3192109629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91b4004d40 0x7f91b4005150 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f91c400e320 tx=0x7f91b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67d340bb1d6ac62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:56.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.814+0000 7f91caffd640 1 -- 172.21.15.69:0/3192109629 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c4033070 con 0x7f91b4004d40 2024-04-25T10:59:56.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.814+0000 7f91caffd640 1 -- 172.21.15.69:0/3192109629 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91c402f660 con 0x7f91b4004d40 2024-04-25T10:59:56.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.814+0000 7f91caffd640 1 -- 172.21.15.69:0/3192109629 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c4009de0 con 0x7f91b4004d40 2024-04-25T10:59:56.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.814+0000 7f91d2734640 1 -- 172.21.15.69:0/3192109629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91b4004d40 msgr2=0x7f91b4005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:56.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.814+0000 7f91d2734640 1 --2- 172.21.15.69:0/3192109629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91b4004d40 0x7f91b4005150 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f91c400e320 tx=0x7f91b4005830 comp rx=0 tx=0).stop 2024-04-25T10:59:56.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.814+0000 7f91d0cd1640 1 -- 172.21.15.69:0/3192109629 reap_dead start 2024-04-25T10:59:56.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.814+0000 7f91d2734640 1 -- 172.21.15.69:0/3192109629 shutdown_connections 2024-04-25T10:59:56.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.814+0000 7f91d2734640 1 -- 172.21.15.69:0/3192109629 >> 172.21.15.69:0/3192109629 conn(0x7f91cc0d3a50 msgr2=0x7f91cc0d3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:56.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.814+0000 7f91d2734640 1 -- 172.21.15.69:0/3192109629 shutdown_connections 2024-04-25T10:59:56.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.814+0000 7f91d2734640 1 -- 172.21.15.69:0/3192109629 wait complete. 2024-04-25T10:59:56.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.816+0000 7f91d2734640 1 Processor -- start 2024-04-25T10:59:56.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.816+0000 7f91d2734640 1 -- start start 2024-04-25T10:59:56.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.817+0000 7f91d2734640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91cc1f3bf0 0x7f91cc15b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:56.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.817+0000 7f91d2734640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91cc15b750 0x7f91cc0b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:56.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.817+0000 7f91d2734640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91cc0b8910 0x7f91cc1f78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:56.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.817+0000 7f91d2734640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f91cc163540 con 0x7f91cc15b750 2024-04-25T10:59:56.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.817+0000 7f91d2734640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f91cc0b51c0 con 0x7f91cc0b8910 2024-04-25T10:59:56.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.817+0000 7f91d2734640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f91cc15c600 con 0x7f91cc1f3bf0 2024-04-25T10:59:56.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.818+0000 7f91cb7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91cc15b750 0x7f91cc0b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:56.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.818+0000 7f91cb7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91cc15b750 0x7f91cc0b83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48056/0 (socket says 172.21.15.69:48056) 2024-04-25T10:59:56.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.818+0000 7f91cb7fe640 1 -- 172.21.15.69:0/790989420 learned_addr learned my addr 172.21.15.69:0/790989420 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:56.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.818+0000 7f91cbfff640 1 --2- 172.21.15.69:0/790989420 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91cc1f3bf0 0x7f91cc15b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:56.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.818+0000 7f91d0cd1640 1 --2- 172.21.15.69:0/790989420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91cc0b8910 0x7f91cc1f78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:56.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.818+0000 7f91cb7fe640 1 -- 172.21.15.69:0/790989420 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91cc1f3bf0 msgr2=0x7f91cc15b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:56.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.818+0000 7f91cb7fe640 1 --2- 172.21.15.69:0/790989420 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91cc1f3bf0 0x7f91cc15b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.818+0000 7f91cb7fe640 1 -- 172.21.15.69:0/790989420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91cc0b8910 msgr2=0x7f91cc1f78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:56.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.818+0000 7f91cb7fe640 1 --2- 172.21.15.69:0/790989420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91cc0b8910 0x7f91cc1f78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:56.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.818+0000 7f91cb7fe640 1 -- 172.21.15.69:0/790989420 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91cc0b8d20 con 0x7f91cc15b750 2024-04-25T10:59:56.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.818+0000 7f91cbfff640 1 --2- 172.21.15.69:0/790989420 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91cc1f3bf0 0x7f91cc15b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:56.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.818+0000 7f91d0cd1640 1 --2- 172.21.15.69:0/790989420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91cc0b8910 0x7f91cc1f78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:56.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.818+0000 7f91cb7fe640 1 --2- 172.21.15.69:0/790989420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91cc15b750 0x7f91cc0b83d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f91c0004840 tx=0x7f91c0007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:56.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.818+0000 7f91c8ff9640 1 -- 172.21.15.69:0/790989420 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c000f9c0 con 0x7f91cc15b750 2024-04-25T10:59:56.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.818+0000 7f91c8ff9640 1 -- 172.21.15.69:0/790989420 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91c0018070 con 0x7f91cc15b750 2024-04-25T10:59:56.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.818+0000 7f91c8ff9640 1 -- 172.21.15.69:0/790989420 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c000fda0 con 0x7f91cc15b750 2024-04-25T10:59:56.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.818+0000 7f91d2734640 1 -- 172.21.15.69:0/790989420 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91cc15a560 con 0x7f91cc15b750 2024-04-25T10:59:56.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.818+0000 7f91d2734640 1 -- 172.21.15.69:0/790989420 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91cc15c040 con 0x7f91cc15b750 2024-04-25T10:59:56.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.819+0000 7f91d2734640 1 -- 172.21.15.69:0/790989420 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91cc050740 con 0x7f91cc15b750 2024-04-25T10:59:56.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.820+0000 7f91c8ff9640 1 -- 172.21.15.69:0/790989420 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f91c001d460 con 0x7f91cc15b750 2024-04-25T10:59:56.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.820+0000 7f91c8ff9640 1 --2- 172.21.15.69:0/790989420 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f91b80744b0 0x7f91b80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:56.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.820+0000 7f91c8ff9640 1 -- 172.21.15.69:0/790989420 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f91c0095330 con 0x7f91cc15b750 2024-04-25T10:59:56.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.820+0000 7f91cbfff640 1 --2- 172.21.15.69:0/790989420 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f91b80744b0 0x7f91b80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:56.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.821+0000 7f91cbfff640 1 --2- 172.21.15.69:0/790989420 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f91b80744b0 0x7f91b80769c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f91bc009c30 tx=0x7f91bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:56.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:56.825+0000 7f91c8ff9640 1 -- 172.21.15.69:0/790989420 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f91cc0b8d20 con 0x7f91cc15b750 2024-04-25T10:59:57.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.037+0000 7f91d2734640 1 -- 172.21.15.69:0/790989420 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f91cc0d1220 con 0x7f91cc15b750 2024-04-25T10:59:57.038 INFO:tasks.ceph.mon.a.smithi069.stderr:2024-04-25T10:59:57.037+0000 7f5831b98640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:59:57.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.039+0000 7f91c8ff9640 1 -- 172.21.15.69:0/790989420 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) ==== 161+0+0 (secure 0 0 0) 0x7f91c005fb50 con 0x7f91cc15b750 2024-04-25T10:59:57.039 INFO:teuthology.orchestra.run.smithi069.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:59:57.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.041+0000 7f91d2734640 1 -- 172.21.15.69:0/790989420 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f91b80744b0 msgr2=0x7f91b80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:57.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.041+0000 7f91d2734640 1 --2- 172.21.15.69:0/790989420 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f91b80744b0 0x7f91b80769c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f91bc009c30 tx=0x7f91bc008040 comp rx=0 tx=0).stop 2024-04-25T10:59:57.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.041+0000 7f91d2734640 1 -- 172.21.15.69:0/790989420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91cc15b750 msgr2=0x7f91cc0b83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:57.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.041+0000 7f91d2734640 1 --2- 172.21.15.69:0/790989420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91cc15b750 0x7f91cc0b83d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f91c0004840 tx=0x7f91c0007720 comp rx=0 tx=0).stop 2024-04-25T10:59:57.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.041+0000 7f91d2734640 1 -- 172.21.15.69:0/790989420 shutdown_connections 2024-04-25T10:59:57.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.041+0000 7f91d2734640 1 --2- 172.21.15.69:0/790989420 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f91b80744b0 0x7f91b80769c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.041+0000 7f91d2734640 1 --2- 172.21.15.69:0/790989420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91cc0b8910 0x7f91cc1f78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.041+0000 7f91d2734640 1 --2- 172.21.15.69:0/790989420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91cc15b750 0x7f91cc0b83d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.041+0000 7f91d2734640 1 --2- 172.21.15.69:0/790989420 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91cc1f3bf0 0x7f91cc15b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.041+0000 7f91d2734640 1 -- 172.21.15.69:0/790989420 >> 172.21.15.69:0/790989420 conn(0x7f91cc0d3a50 msgr2=0x7f91cc15d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:57.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.041+0000 7f91d2734640 1 -- 172.21.15.69:0/790989420 shutdown_connections 2024-04-25T10:59:57.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.041+0000 7f91d2734640 1 -- 172.21.15.69:0/790989420 wait complete. 2024-04-25T10:59:57.047 INFO:tasks.ceph.mds.b.smithi146.stderr: -7355> 2024-04-25T10:59:23.234+0000 7faf039d1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:59:57.054 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:59:57.103 INFO:tasks.ceph.mds.b.smithi146.stdout:starting mds.b at 2024-04-25T10:59:57.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.164+0000 7f7c12882640 1 Processor -- start 2024-04-25T10:59:57.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.164+0000 7f7c12882640 1 -- start start 2024-04-25T10:59:57.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.165+0000 7f7c12882640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c0c143660 0x7f7c0c070290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:57.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.165+0000 7f7c12882640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7c0c0b56e0 0x7f7c0c0b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:57.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.165+0000 7f7c12882640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c0c070c70 con 0x7f7c0c06d5d0 2024-04-25T10:59:57.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.165+0000 7f7c12882640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f7c0c0b4920 con 0x7f7c0c0b56e0 2024-04-25T10:59:57.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.165+0000 7f7c12882640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7c0c0b4c00 con 0x7f7c0c143660 2024-04-25T10:59:57.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.166+0000 7f7c10e1f640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7c0c0b56e0 0x7f7c0c0b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:57.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.166+0000 7f7c10e1f640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7c0c0b56e0 0x7f7c0c0b5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48062/0 (socket says 172.21.15.69:48062) 2024-04-25T10:59:57.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.166+0000 7f7c10e1f640 1 -- 172.21.15.69:0/1217753695 learned_addr learned my addr 172.21.15.69:0/1217753695 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:57.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.166+0000 7f7c0bfff640 1 --2- 172.21.15.69:0/1217753695 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c0c143660 0x7f7c0c070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:57.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.166+0000 7f7c10e1f640 1 -- 172.21.15.69:0/1217753695 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c0c143660 msgr2=0x7f7c0c070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:57.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.166+0000 7f7c10e1f640 1 --2- 172.21.15.69:0/1217753695 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c0c143660 0x7f7c0c070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.166+0000 7f7c10e1f640 1 -- 172.21.15.69:0/1217753695 >> v1:172.21.15.146:6789/0 conn(0x7f7c0c06d5d0 legacy=0x7f7c0c06d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:57.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.166+0000 7f7c0affd640 1 -- 172.21.15.69:0/1217753695 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2147068865 0 0) 0x7f7c0c070c70 con 0x7f7c0c06d5d0 2024-04-25T10:59:57.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.166+0000 7f7c10e1f640 1 -- 172.21.15.69:0/1217753695 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0c0b4ee0 con 0x7f7c0c0b56e0 2024-04-25T10:59:57.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.167+0000 7f7c10e1f640 1 --2- 172.21.15.69:0/1217753695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c0c0b56e0 0x7f7c0c0b5af0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400d150 tx=0x7f7c0402c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f3140aaf867aef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:57.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.167+0000 7f7c0affd640 1 -- 172.21.15.69:0/1217753695 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04009600 con 0x7f7c0c0b56e0 2024-04-25T10:59:57.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.167+0000 7f7c0affd640 1 -- 172.21.15.69:0/1217753695 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c04037070 con 0x7f7c0c0b56e0 2024-04-25T10:59:57.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.167+0000 7f7c0affd640 1 -- 172.21.15.69:0/1217753695 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c040099c0 con 0x7f7c0c0b56e0 2024-04-25T10:59:57.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.168+0000 7f7c12882640 1 -- 172.21.15.69:0/1217753695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c0c0b56e0 msgr2=0x7f7c0c0b5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:57.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.168+0000 7f7c12882640 1 --2- 172.21.15.69:0/1217753695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c0c0b56e0 0x7f7c0c0b5af0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400d150 tx=0x7f7c0402c930 comp rx=0 tx=0).stop 2024-04-25T10:59:57.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.168+0000 7f7c12882640 1 -- 172.21.15.69:0/1217753695 shutdown_connections 2024-04-25T10:59:57.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.168+0000 7f7c12882640 1 --2- 172.21.15.69:0/1217753695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c0c0b56e0 0x7f7c0c0b5af0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.168+0000 7f7c12882640 1 --2- 172.21.15.69:0/1217753695 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c0c143660 0x7f7c0c070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.168+0000 7f7c12882640 1 -- 172.21.15.69:0/1217753695 >> 172.21.15.69:0/1217753695 conn(0x7f7c0c073120 msgr2=0x7f7c0c073530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:57.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.168+0000 7f7c12882640 1 -- 172.21.15.69:0/1217753695 shutdown_connections 2024-04-25T10:59:57.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.168+0000 7f7c12882640 1 -- 172.21.15.69:0/1217753695 wait complete. 2024-04-25T10:59:57.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.170+0000 7f7c12882640 1 Processor -- start 2024-04-25T10:59:57.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.170+0000 7f7c12882640 1 -- start start 2024-04-25T10:59:57.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.171+0000 7f7c12882640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c0c06d5d0 0x7f7c0c1e2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:57.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.171+0000 7f7c12882640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c0c0b56e0 0x7f7c0c14c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:57.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.171+0000 7f7c12882640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c0c143660 0x7f7c0c1f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:57.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.171+0000 7f7c12882640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c0ce260 con 0x7f7c0c143660 2024-04-25T10:59:57.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.171+0000 7f7c12882640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c0ce0e0 con 0x7f7c0c06d5d0 2024-04-25T10:59:57.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.171+0000 7f7c12882640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7c0c0707d0 con 0x7f7c0c0b56e0 2024-04-25T10:59:57.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.172+0000 7f7c10e1f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c0c143660 0x7f7c0c1f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:57.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.172+0000 7f7c10e1f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c0c143660 0x7f7c0c1f4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48070/0 (socket says 172.21.15.69:48070) 2024-04-25T10:59:57.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.172+0000 7f7c10e1f640 1 -- 172.21.15.69:0/1568924479 learned_addr learned my addr 172.21.15.69:0/1568924479 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:57.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.172+0000 7f7c0bfff640 1 --2- 172.21.15.69:0/1568924479 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c0c06d5d0 0x7f7c0c1e2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:57.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.172+0000 7f7c0b7fe640 1 --2- 172.21.15.69:0/1568924479 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c0c0b56e0 0x7f7c0c14c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:57.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.172+0000 7f7c10e1f640 1 -- 172.21.15.69:0/1568924479 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c0c0b56e0 msgr2=0x7f7c0c14c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:57.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.172+0000 7f7c10e1f640 1 --2- 172.21.15.69:0/1568924479 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c0c0b56e0 0x7f7c0c14c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.172+0000 7f7c10e1f640 1 -- 172.21.15.69:0/1568924479 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c0c06d5d0 msgr2=0x7f7c0c1e2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:57.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.172+0000 7f7c10e1f640 1 --2- 172.21.15.69:0/1568924479 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c0c06d5d0 0x7f7c0c1e2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:57.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.172+0000 7f7c10e1f640 1 -- 172.21.15.69:0/1568924479 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0c1e29a0 con 0x7f7c0c143660 2024-04-25T10:59:57.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.172+0000 7f7c10e1f640 1 --2- 172.21.15.69:0/1568924479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c0c143660 0x7f7c0c1f4a00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7c04034040 tx=0x7f7c04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:57.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.172+0000 7f7c08ff9640 1 -- 172.21.15.69:0/1568924479 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04009dc0 con 0x7f7c0c143660 2024-04-25T10:59:57.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.172+0000 7f7c08ff9640 1 -- 172.21.15.69:0/1568924479 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c04047480 con 0x7f7c0c143660 2024-04-25T10:59:57.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.172+0000 7f7c12882640 1 -- 172.21.15.69:0/1568924479 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c0c1e2c70 con 0x7f7c0c143660 2024-04-25T10:59:57.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.172+0000 7f7c08ff9640 1 -- 172.21.15.69:0/1568924479 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04047870 con 0x7f7c0c143660 2024-04-25T10:59:57.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.172+0000 7f7c12882640 1 -- 172.21.15.69:0/1568924479 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c0c1e3230 con 0x7f7c0c143660 2024-04-25T10:59:57.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.173+0000 7f7c12882640 1 -- 172.21.15.69:0/1568924479 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c0c050740 con 0x7f7c0c143660 2024-04-25T10:59:57.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.175+0000 7f7c08ff9640 1 -- 172.21.15.69:0/1568924479 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7c04047a10 con 0x7f7c0c143660 2024-04-25T10:59:57.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.175+0000 7f7c08ff9640 1 --2- 172.21.15.69:0/1568924479 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f7bf80744b0 0x7f7bf80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:57.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.175+0000 7f7c08ff9640 1 -- 172.21.15.69:0/1568924479 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5795+0+0 (secure 0 0 0) 0x7f7c040b8660 con 0x7f7c0c143660 2024-04-25T10:59:57.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.175+0000 7f7c0bfff640 1 --2- 172.21.15.69:0/1568924479 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f7bf80744b0 0x7f7bf80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:57.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.176+0000 7f7c0bfff640 1 --2- 172.21.15.69:0/1568924479 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f7bf80744b0 0x7f7bf80769c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c0cfca0 tx=0x7f7c00009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:57.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.180+0000 7f7c08ff9640 1 -- 172.21.15.69:0/1568924479 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7c04082740 con 0x7f7c0c143660 2024-04-25T10:59:57.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:57.398+0000 7f7c12882640 1 -- 172.21.15.69:0/1568924479 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f7c0c141e20 con 0x7f7c0c143660 2024-04-25T10:59:58.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.040+0000 7f7c08ff9640 1 -- 172.21.15.69:0/1568924479 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) ==== 104+0+0 (secure 0 0 0) 0x7f7c04086a40 con 0x7f7c0c143660 2024-04-25T10:59:58.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.043+0000 7f7c12882640 1 -- 172.21.15.69:0/1568924479 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f7bf80744b0 msgr2=0x7f7bf80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:58.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.043+0000 7f7c12882640 1 --2- 172.21.15.69:0/1568924479 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f7bf80744b0 0x7f7bf80769c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c0cfca0 tx=0x7f7c00009190 comp rx=0 tx=0).stop 2024-04-25T10:59:58.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.043+0000 7f7c12882640 1 -- 172.21.15.69:0/1568924479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c0c143660 msgr2=0x7f7c0c1f4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:58.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.043+0000 7f7c12882640 1 --2- 172.21.15.69:0/1568924479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c0c143660 0x7f7c0c1f4a00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7c04034040 tx=0x7f7c04004a40 comp rx=0 tx=0).stop 2024-04-25T10:59:58.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.043+0000 7f7c12882640 1 -- 172.21.15.69:0/1568924479 shutdown_connections 2024-04-25T10:59:58.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.043+0000 7f7c12882640 1 --2- 172.21.15.69:0/1568924479 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f7bf80744b0 0x7f7bf80769c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:58.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.043+0000 7f7c12882640 1 --2- 172.21.15.69:0/1568924479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c0c143660 0x7f7c0c1f4a00 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:58.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.043+0000 7f7c12882640 1 --2- 172.21.15.69:0/1568924479 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c0c0b56e0 0x7f7c0c14c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:58.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.043+0000 7f7c12882640 1 --2- 172.21.15.69:0/1568924479 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c0c06d5d0 0x7f7c0c1e2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:58.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.043+0000 7f7c12882640 1 -- 172.21.15.69:0/1568924479 >> 172.21.15.69:0/1568924479 conn(0x7f7c0c073120 msgr2=0x7f7c0c0b6330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:58.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.043+0000 7f7c12882640 1 -- 172.21.15.69:0/1568924479 shutdown_connections 2024-04-25T10:59:58.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.043+0000 7f7c12882640 1 -- 172.21.15.69:0/1568924479 wait complete. 2024-04-25T10:59:58.059 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:59:58.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.172+0000 7f307b38f640 1 Processor -- start 2024-04-25T10:59:58.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.172+0000 7f307b38f640 1 -- start start 2024-04-25T10:59:58.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.174+0000 7f307b38f640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f307415e080 0x7f30741646b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:58.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.174+0000 7f307b38f640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30741609b0 con 0x7f3074161710 2024-04-25T10:59:58.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.174+0000 7f307b38f640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3074164cd0 con 0x7f307406ccd0 2024-04-25T10:59:58.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.174+0000 7f307b38f640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f3074160c30 con 0x7f307415e080 2024-04-25T10:59:58.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.174+0000 7f307912b640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f3074161710 0x7f307406c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:34744/0 (socket says 172.21.15.69:34744) 2024-04-25T10:59:58.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.174+0000 7f307912b640 1 -- 172.21.15.69:0/1232058281 learned_addr learned my addr 172.21.15.69:0/1232058281 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T10:59:58.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.174+0000 7f307992c640 1 --2- 172.21.15.69:0/1232058281 >> v2:172.21.15.146:3300/0 conn(0x7f307415e080 0x7f30741646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:58.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.174+0000 7f306bfff640 1 -- 172.21.15.69:0/1232058281 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 456635183 0 0) 0x7f3064001110 con 0x7f3074161710 2024-04-25T10:59:58.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.174+0000 7f306bfff640 1 -- 172.21.15.69:0/1232058281 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3060003520 con 0x7f3074161710 2024-04-25T10:59:58.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.174+0000 7f306bfff640 1 -- 172.21.15.69:0/1232058281 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 842775550 0 0) 0x7f3060003520 con 0x7f3074161710 2024-04-25T10:59:58.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.174+0000 7f306bfff640 1 -- 172.21.15.69:0/1232058281 >> v2:172.21.15.146:3300/0 conn(0x7f307415e080 msgr2=0x7f30741646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:58.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.174+0000 7f306bfff640 1 --2- 172.21.15.69:0/1232058281 >> v2:172.21.15.146:3300/0 conn(0x7f307415e080 0x7f30741646b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:58.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.174+0000 7f306bfff640 1 -- 172.21.15.69:0/1232058281 >> v1:172.21.15.146:6789/0 conn(0x7f307406ccd0 legacy=0x7f3074163fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:58.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.174+0000 7f306bfff640 1 -- 172.21.15.69:0/1232058281 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3074160f10 con 0x7f3074161710 2024-04-25T10:59:58.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.174+0000 7f306bfff640 1 -- 172.21.15.69:0/1232058281 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7f3064001840 con 0x7f3074161710 2024-04-25T10:59:58.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.175+0000 7f306bfff640 1 -- 172.21.15.69:0/1232058281 >> v1:172.21.15.69:6789/0 conn(0x7f3074161710 legacy=0x7f307406c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:58.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.175+0000 7f306bfff640 1 --2- 172.21.15.69:0/1232058281 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30600039b0 0x7f3060003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:58.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.175+0000 7f306bfff640 1 --2- 172.21.15.69:0/1232058281 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3060004800 0x7f3060004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:58.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.175+0000 7f306bfff640 1 --2- 172.21.15.69:0/1232058281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30600051c0 0x7f306000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:58.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.175+0000 7f306bfff640 1 -- 172.21.15.69:0/1232058281 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3074160c30 con 0x7f30600051c0 2024-04-25T10:59:58.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.175+0000 7f306bfff640 1 -- 172.21.15.69:0/1232058281 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f306000c160 con 0x7f3060004800 2024-04-25T10:59:58.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.175+0000 7f306bfff640 1 -- 172.21.15.69:0/1232058281 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f306000c360 con 0x7f30600039b0 2024-04-25T10:59:58.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.175+0000 7f307992c640 1 --2- 172.21.15.69:0/1232058281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30600051c0 0x7f306000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:58.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.175+0000 7f307892a640 1 --2- 172.21.15.69:0/1232058281 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3060004800 0x7f3060004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:58.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.175+0000 7f307912b640 1 --2- 172.21.15.69:0/1232058281 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30600039b0 0x7f3060003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:58.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.175+0000 7f307992c640 1 -- 172.21.15.69:0/1232058281 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30600039b0 msgr2=0x7f3060003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:58.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.175+0000 7f307992c640 1 --2- 172.21.15.69:0/1232058281 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30600039b0 0x7f3060003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:58.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.175+0000 7f307992c640 1 -- 172.21.15.69:0/1232058281 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3060004800 msgr2=0x7f3060004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:58.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.175+0000 7f307992c640 1 --2- 172.21.15.69:0/1232058281 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3060004800 0x7f3060004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:58.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.175+0000 7f307992c640 1 -- 172.21.15.69:0/1232058281 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3060003520 con 0x7f30600051c0 2024-04-25T10:59:58.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.176+0000 7f307992c640 1 -- 172.21.15.69:0/1232058281 reap_dead start 2024-04-25T10:59:58.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.176+0000 7f307992c640 1 --2- 172.21.15.69:0/1232058281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30600051c0 0x7f306000b9c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f306c009f90 tx=0x7f3060005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e2a911080be948 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:58.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.176+0000 7f306bfff640 1 -- 172.21.15.69:0/1232058281 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c035020 con 0x7f30600051c0 2024-04-25T10:59:58.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.176+0000 7f306bfff640 1 -- 172.21.15.69:0/1232058281 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f306c003b90 con 0x7f30600051c0 2024-04-25T10:59:58.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.176+0000 7f306bfff640 1 -- 172.21.15.69:0/1232058281 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c009c10 con 0x7f30600051c0 2024-04-25T10:59:58.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.177+0000 7f307b38f640 1 -- 172.21.15.69:0/1232058281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30600051c0 msgr2=0x7f306000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:58.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.177+0000 7f307b38f640 1 --2- 172.21.15.69:0/1232058281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30600051c0 0x7f306000b9c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f306c009f90 tx=0x7f3060005830 comp rx=0 tx=0).stop 2024-04-25T10:59:58.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.177+0000 7f307b38f640 1 -- 172.21.15.69:0/1232058281 shutdown_connections 2024-04-25T10:59:58.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.177+0000 7f307b38f640 1 --2- 172.21.15.69:0/1232058281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30600051c0 0x7f306000b9c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:58.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.177+0000 7f307b38f640 1 -- 172.21.15.69:0/1232058281 >> 172.21.15.69:0/1232058281 conn(0x7f30740d5d90 msgr2=0x7f30740d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:58.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.177+0000 7f307b38f640 1 -- 172.21.15.69:0/1232058281 shutdown_connections 2024-04-25T10:59:58.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.177+0000 7f307b38f640 1 -- 172.21.15.69:0/1232058281 wait complete. 2024-04-25T10:59:58.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.179+0000 7f307b38f640 1 Processor -- start 2024-04-25T10:59:58.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.179+0000 7f307b38f640 1 -- start start 2024-04-25T10:59:58.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.180+0000 7f307b38f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30600051c0 0x7f3074157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:58.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.180+0000 7f307b38f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30740cdfb0 0x7f3074153100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:58.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.180+0000 7f307b38f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30740ba3f0 0x7f30740ba800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:58.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.180+0000 7f307b38f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f307406c740 con 0x7f30600051c0 2024-04-25T10:59:58.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.180+0000 7f307b38f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f307406c5c0 con 0x7f30740ba3f0 2024-04-25T10:59:58.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.180+0000 7f307b38f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f307406c8c0 con 0x7f30740cdfb0 2024-04-25T10:59:58.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.180+0000 7f307912b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30600051c0 0x7f3074157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:58.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.180+0000 7f307912b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30600051c0 0x7f3074157150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48086/0 (socket says 172.21.15.69:48086) 2024-04-25T10:59:58.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.180+0000 7f307912b640 1 -- 172.21.15.69:0/1765254252 learned_addr learned my addr 172.21.15.69:0/1765254252 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:58.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.180+0000 7f307892a640 1 --2- 172.21.15.69:0/1765254252 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30740cdfb0 0x7f3074153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:58.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.180+0000 7f307992c640 1 --2- 172.21.15.69:0/1765254252 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30740ba3f0 0x7f30740ba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:58.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.180+0000 7f307912b640 1 -- 172.21.15.69:0/1765254252 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30740cdfb0 msgr2=0x7f3074153100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:58.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.180+0000 7f307912b640 1 --2- 172.21.15.69:0/1765254252 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30740cdfb0 0x7f3074153100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:58.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.180+0000 7f307912b640 1 -- 172.21.15.69:0/1765254252 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30740ba3f0 msgr2=0x7f30740ba800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:58.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.180+0000 7f307912b640 1 --2- 172.21.15.69:0/1765254252 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30740ba3f0 0x7f30740ba800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:58.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.181+0000 7f307912b640 1 -- 172.21.15.69:0/1765254252 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30740bad40 con 0x7f30600051c0 2024-04-25T10:59:58.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.181+0000 7f307912b640 1 --2- 172.21.15.69:0/1765254252 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30600051c0 0x7f3074157150 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3064004e40 tx=0x7f3064007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:58.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.181+0000 7f3069ffb640 1 -- 172.21.15.69:0/1765254252 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306400f7b0 con 0x7f30600051c0 2024-04-25T10:59:58.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.181+0000 7f3069ffb640 1 -- 172.21.15.69:0/1765254252 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3064018070 con 0x7f30600051c0 2024-04-25T10:59:58.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.181+0000 7f307b38f640 1 -- 172.21.15.69:0/1765254252 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3074159a00 con 0x7f30600051c0 2024-04-25T10:59:58.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.181+0000 7f3069ffb640 1 -- 172.21.15.69:0/1765254252 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306400fbb0 con 0x7f30600051c0 2024-04-25T10:59:58.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.181+0000 7f307b38f640 1 -- 172.21.15.69:0/1765254252 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3074159fc0 con 0x7f30600051c0 2024-04-25T10:59:58.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.182+0000 7f307b38f640 1 -- 172.21.15.69:0/1765254252 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3074050740 con 0x7f30600051c0 2024-04-25T10:59:58.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.183+0000 7f3069ffb640 1 -- 172.21.15.69:0/1765254252 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f306400fd50 con 0x7f30600051c0 2024-04-25T10:59:58.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.183+0000 7f3069ffb640 1 --2- 172.21.15.69:0/1765254252 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f30580744e0 0x7f30580769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:58.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.184+0000 7f307892a640 1 --2- 172.21.15.69:0/1765254252 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f30580744e0 0x7f30580769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:58.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.184+0000 7f3069ffb640 1 -- 172.21.15.69:0/1765254252 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5795+0+0 (secure 0 0 0) 0x7f3064095170 con 0x7f30600051c0 2024-04-25T10:59:58.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.184+0000 7f307892a640 1 --2- 172.21.15.69:0/1765254252 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f30580744e0 0x7f30580769f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f30741f8ef0 tx=0x7f3070009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:58.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.188+0000 7f3069ffb640 1 -- 172.21.15.69:0/1765254252 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f30740bad40 con 0x7f30600051c0 2024-04-25T10:59:58.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:58.366+0000 7f307b38f640 1 -- 172.21.15.69:0/1765254252 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f30740d15d0 con 0x7f30600051c0 2024-04-25T10:59:59.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.045+0000 7f3069ffb640 1 -- 172.21.15.69:0/1765254252 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v22) ==== 181+0+0 (secure 0 0 0) 0x7f30741609b0 con 0x7f30600051c0 2024-04-25T10:59:59.046 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:59:59.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.048+0000 7f307b38f640 1 -- 172.21.15.69:0/1765254252 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f30580744e0 msgr2=0x7f30580769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:59.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.048+0000 7f307b38f640 1 --2- 172.21.15.69:0/1765254252 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f30580744e0 0x7f30580769f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f30741f8ef0 tx=0x7f3070009040 comp rx=0 tx=0).stop 2024-04-25T10:59:59.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.048+0000 7f307b38f640 1 -- 172.21.15.69:0/1765254252 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30600051c0 msgr2=0x7f3074157150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:59.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.048+0000 7f307b38f640 1 --2- 172.21.15.69:0/1765254252 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30600051c0 0x7f3074157150 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3064004e40 tx=0x7f3064007430 comp rx=0 tx=0).stop 2024-04-25T10:59:59.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.048+0000 7f307b38f640 1 -- 172.21.15.69:0/1765254252 shutdown_connections 2024-04-25T10:59:59.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.048+0000 7f307b38f640 1 --2- 172.21.15.69:0/1765254252 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f30580744e0 0x7f30580769f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:59.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.048+0000 7f307b38f640 1 --2- 172.21.15.69:0/1765254252 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30740ba3f0 0x7f30740ba800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:59.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.048+0000 7f307b38f640 1 --2- 172.21.15.69:0/1765254252 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30740cdfb0 0x7f3074153100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:59.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.048+0000 7f307b38f640 1 --2- 172.21.15.69:0/1765254252 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30600051c0 0x7f3074157150 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:59.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.048+0000 7f307b38f640 1 -- 172.21.15.69:0/1765254252 >> 172.21.15.69:0/1765254252 conn(0x7f30740d5d90 msgr2=0x7f307415e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:59.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.048+0000 7f307b38f640 1 -- 172.21.15.69:0/1765254252 shutdown_connections 2024-04-25T10:59:59.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.048+0000 7f307b38f640 1 -- 172.21.15.69:0/1765254252 wait complete. 2024-04-25T10:59:59.066 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:59:59.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.179+0000 7f3782e51640 1 Processor -- start 2024-04-25T10:59:59.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.179+0000 7f3782e51640 1 -- start start 2024-04-25T10:59:59.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.180+0000 7f3782e51640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f377c0d0e00 0x7f377c0d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:59.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.180+0000 7f3782e51640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f377c06c5c0 0x7f377c06c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:59.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.180+0000 7f3782e51640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f377c06cf10 con 0x7f377c0b7810 2024-04-25T10:59:59.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.180+0000 7f3782e51640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f377c0d5a60 con 0x7f377c06c5c0 2024-04-25T10:59:59.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.180+0000 7f3782e51640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f377c0d1850 con 0x7f377c0d0e00 2024-04-25T10:59:59.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.180+0000 7f377bfff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f377c06c5c0 0x7f377c06c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:59.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.180+0000 7f377bfff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f377c06c5c0 0x7f377c06c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48094/0 (socket says 172.21.15.69:48094) 2024-04-25T10:59:59.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.180+0000 7f377bfff640 1 -- 172.21.15.69:0/1900706684 learned_addr learned my addr 172.21.15.69:0/1900706684 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:59.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.180+0000 7f3780bed640 1 --2- 172.21.15.69:0/1900706684 >> v2:172.21.15.146:3300/0 conn(0x7f377c0d0e00 0x7f377c0d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:59.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.181+0000 7f377bfff640 1 -- 172.21.15.69:0/1900706684 >> v2:172.21.15.146:3300/0 conn(0x7f377c0d0e00 msgr2=0x7f377c0d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:59.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.181+0000 7f377bfff640 1 --2- 172.21.15.69:0/1900706684 >> v2:172.21.15.146:3300/0 conn(0x7f377c0d0e00 0x7f377c0d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:59.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.181+0000 7f377bfff640 1 -- 172.21.15.69:0/1900706684 >> v1:172.21.15.146:6789/0 conn(0x7f377c0b7810 legacy=0x7f377c0b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:59:59.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.181+0000 7f377bfff640 1 -- 172.21.15.69:0/1900706684 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377c160de0 con 0x7f377c06c5c0 2024-04-25T10:59:59.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.181+0000 7f3780bed640 1 --2- 172.21.15.69:0/1900706684 >> v2:172.21.15.146:3300/0 conn(0x7f377c0d0e00 0x7f377c0d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:59:59.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.181+0000 7f377bfff640 1 --2- 172.21.15.69:0/1900706684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f377c06c5c0 0x7f377c06c9d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f377c0d5be0 tx=0x7f376c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f69a1ba624b5bf12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:59.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.181+0000 7f377b7fe640 1 -- 172.21.15.69:0/1900706684 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c037070 con 0x7f377c06c5c0 2024-04-25T10:59:59.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.181+0000 7f377b7fe640 1 -- 172.21.15.69:0/1900706684 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f376c032440 con 0x7f377c06c5c0 2024-04-25T10:59:59.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.181+0000 7f377b7fe640 1 -- 172.21.15.69:0/1900706684 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c032810 con 0x7f377c06c5c0 2024-04-25T10:59:59.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.182+0000 7f3782e51640 1 -- 172.21.15.69:0/1900706684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f377c06c5c0 msgr2=0x7f377c06c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:59.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.182+0000 7f3782e51640 1 --2- 172.21.15.69:0/1900706684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f377c06c5c0 0x7f377c06c9d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f377c0d5be0 tx=0x7f376c02a180 comp rx=0 tx=0).stop 2024-04-25T10:59:59.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.182+0000 7f3782e51640 1 -- 172.21.15.69:0/1900706684 shutdown_connections 2024-04-25T10:59:59.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.182+0000 7f3782e51640 1 --2- 172.21.15.69:0/1900706684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f377c06c5c0 0x7f377c06c9d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:59.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.182+0000 7f3782e51640 1 --2- 172.21.15.69:0/1900706684 >> v2:172.21.15.146:3300/0 conn(0x7f377c0d0e00 0x7f377c0d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:59.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.182+0000 7f3782e51640 1 -- 172.21.15.69:0/1900706684 >> 172.21.15.69:0/1900706684 conn(0x7f377c0d5d90 msgr2=0x7f377c0d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:59:59.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.182+0000 7f3782e51640 1 -- 172.21.15.69:0/1900706684 shutdown_connections 2024-04-25T10:59:59.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.182+0000 7f3782e51640 1 -- 172.21.15.69:0/1900706684 wait complete. 2024-04-25T10:59:59.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.184+0000 7f3782e51640 1 Processor -- start 2024-04-25T10:59:59.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.184+0000 7f3782e51640 1 -- start start 2024-04-25T10:59:59.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.185+0000 7f3782e51640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f377c06c5c0 0x7f377c0ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:59.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.185+0000 7f3782e51640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f377c0b7810 0x7f377c1f77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:59.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.185+0000 7f3782e51640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f377c0d0e00 0x7f377c1f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:59.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.185+0000 7f3782e51640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f377c160ad0 con 0x7f377c0b7810 2024-04-25T10:59:59.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.185+0000 7f3782e51640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f377c1613a0 con 0x7f377c0d0e00 2024-04-25T10:59:59.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.185+0000 7f3782e51640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f377c0b5320 con 0x7f377c06c5c0 2024-04-25T10:59:59.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.185+0000 7f377bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f377c0b7810 0x7f377c1f77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:59.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.185+0000 7f377bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f377c0b7810 0x7f377c1f77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48104/0 (socket says 172.21.15.69:48104) 2024-04-25T10:59:59.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.185+0000 7f377bfff640 1 -- 172.21.15.69:0/2521180905 learned_addr learned my addr 172.21.15.69:0/2521180905 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T10:59:59.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.186+0000 7f37813ee640 1 --2- 172.21.15.69:0/2521180905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f377c0d0e00 0x7f377c1f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:59.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.186+0000 7f377bfff640 1 -- 172.21.15.69:0/2521180905 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f377c06c5c0 msgr2=0x7f377c0ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:59.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.186+0000 7f377bfff640 1 --2- 172.21.15.69:0/2521180905 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f377c06c5c0 0x7f377c0ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:59.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.186+0000 7f377bfff640 1 -- 172.21.15.69:0/2521180905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f377c0d0e00 msgr2=0x7f377c1f41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:59:59.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.186+0000 7f377bfff640 1 --2- 172.21.15.69:0/2521180905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f377c0d0e00 0x7f377c1f41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:59:59.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.186+0000 7f377bfff640 1 -- 172.21.15.69:0/2521180905 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377c1f7030 con 0x7f377c0b7810 2024-04-25T10:59:59.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.186+0000 7f377bfff640 1 --2- 172.21.15.69:0/2521180905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f377c0b7810 0x7f377c1f77d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f376c000c00 tx=0x7f376c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:59.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.186+0000 7f37797fa640 1 -- 172.21.15.69:0/2521180905 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c037040 con 0x7f377c0b7810 2024-04-25T10:59:59.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.186+0000 7f37797fa640 1 -- 172.21.15.69:0/2521180905 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f376c042070 con 0x7f377c0b7810 2024-04-25T10:59:59.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.186+0000 7f3782e51640 1 -- 172.21.15.69:0/2521180905 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f377c1fae00 con 0x7f377c0b7810 2024-04-25T10:59:59.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.186+0000 7f37797fa640 1 -- 172.21.15.69:0/2521180905 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c030bb0 con 0x7f377c0b7810 2024-04-25T10:59:59.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.186+0000 7f3782e51640 1 -- 172.21.15.69:0/2521180905 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f377c1fb3c0 con 0x7f377c0b7810 2024-04-25T10:59:59.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.187+0000 7f3782e51640 1 -- 172.21.15.69:0/2521180905 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f377c050740 con 0x7f377c0b7810 2024-04-25T10:59:59.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.188+0000 7f37797fa640 1 -- 172.21.15.69:0/2521180905 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f376c047460 con 0x7f377c0b7810 2024-04-25T10:59:59.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.188+0000 7f37797fa640 1 --2- 172.21.15.69:0/2521180905 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f375c0744b0 0x7f375c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:59:59.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.188+0000 7f37797fa640 1 -- 172.21.15.69:0/2521180905 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5473+0+0 (secure 0 0 0) 0x7f376c0b5c80 con 0x7f377c0b7810 2024-04-25T10:59:59.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.188+0000 7f3780bed640 1 --2- 172.21.15.69:0/2521180905 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f375c0744b0 0x7f375c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:59:59.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.189+0000 7f3780bed640 1 --2- 172.21.15.69:0/2521180905 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f375c0744b0 0x7f375c0769c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3764006fd0 tx=0x7f3764008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:59:59.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.193+0000 7f37797fa640 1 -- 172.21.15.69:0/2521180905 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f377c1f7030 con 0x7f377c0b7810 2024-04-25T10:59:59.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T10:59:59.367+0000 7f3782e51640 1 -- 172.21.15.69:0/2521180905 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f377c0d4320 con 0x7f377c0b7810 2024-04-25T11:00:00.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.051+0000 7f37797fa640 1 -- 172.21.15.69:0/2521180905 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v23) ==== 169+0+0 (secure 0 0 0) 0x7f377c06cf10 con 0x7f377c0b7810 2024-04-25T11:00:00.052 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_data' removed 2024-04-25T11:00:00.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.055+0000 7f3782e51640 1 -- 172.21.15.69:0/2521180905 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f375c0744b0 msgr2=0x7f375c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:00.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.055+0000 7f3782e51640 1 --2- 172.21.15.69:0/2521180905 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f375c0744b0 0x7f375c0769c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3764006fd0 tx=0x7f3764008040 comp rx=0 tx=0).stop 2024-04-25T11:00:00.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.055+0000 7f3782e51640 1 -- 172.21.15.69:0/2521180905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f377c0b7810 msgr2=0x7f377c1f77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:00.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.055+0000 7f3782e51640 1 --2- 172.21.15.69:0/2521180905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f377c0b7810 0x7f377c1f77d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f376c000c00 tx=0x7f376c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:00:00.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.056+0000 7f3782e51640 1 -- 172.21.15.69:0/2521180905 shutdown_connections 2024-04-25T11:00:00.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.056+0000 7f3782e51640 1 --2- 172.21.15.69:0/2521180905 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f375c0744b0 0x7f375c0769c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:00.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.056+0000 7f3782e51640 1 --2- 172.21.15.69:0/2521180905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f377c0d0e00 0x7f377c1f41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:00.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.056+0000 7f3782e51640 1 --2- 172.21.15.69:0/2521180905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f377c0b7810 0x7f377c1f77d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:00.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.056+0000 7f3782e51640 1 --2- 172.21.15.69:0/2521180905 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f377c06c5c0 0x7f377c0ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:00.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.056+0000 7f3782e51640 1 -- 172.21.15.69:0/2521180905 >> 172.21.15.69:0/2521180905 conn(0x7f377c0d5d90 msgr2=0x7f377c1641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:00.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.056+0000 7f3782e51640 1 -- 172.21.15.69:0/2521180905 shutdown_connections 2024-04-25T11:00:00.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.056+0000 7f3782e51640 1 -- 172.21.15.69:0/2521180905 wait complete. 2024-04-25T11:00:00.075 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T11:00:00.075 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T11:00:00.075 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T11:00:00.075 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:00:00.075 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T11:00:00.076 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T11:00:00.076 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T11:00:00.076 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:00:00.076 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T11:00:00.076 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T11:00:00.076 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T11:00:00.076 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:00:00.076 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T11:00:00.077 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T11:00:00.077 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T11:00:00.077 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:00:00.085 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T11:00:00.085 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T11:00:00.086 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T11:00:00.086 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T11:00:00.089 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T11:00:00.089 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T11:00:00.091 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T11:00:00.091 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T11:00:00.128 INFO:tasks.ceph.mds.a:Started 2024-04-25T11:00:00.128 DEBUG:teuthology.parallel:result is None 2024-04-25T11:00:00.129 INFO:tasks.ceph.mds.c:Started 2024-04-25T11:00:00.129 DEBUG:teuthology.parallel:result is None 2024-04-25T11:00:00.133 INFO:tasks.ceph.mds.d:Started 2024-04-25T11:00:00.133 DEBUG:teuthology.parallel:result is None 2024-04-25T11:00:00.134 INFO:tasks.ceph.mds.b:Started 2024-04-25T11:00:00.135 DEBUG:teuthology.parallel:result is None 2024-04-25T11:00:00.135 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T11:00:00.214 INFO:tasks.ceph.mds.a.smithi069.stdout:starting mds.a at 2024-04-25T11:00:00.214 INFO:tasks.ceph.mds.c.smithi069.stdout:starting mds.c at 2024-04-25T11:00:00.226 INFO:tasks.ceph.mds.b.smithi146.stdout:starting mds.b at 2024-04-25T11:00:00.227 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2024-04-25T11:00:00.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.252+0000 7f3a3bb3f640 1 Processor -- start 2024-04-25T11:00:00.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.252+0000 7f3a3bb3f640 1 -- start start 2024-04-25T11:00:00.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.253+0000 7f3a3bb3f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a340b8ca0 0x7f3a34163fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:00.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.253+0000 7f3a3bb3f640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3a341644e0 0x7f3a341648f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:00.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.253+0000 7f3a3bb3f640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a3406ed60 con 0x7f3a340d1f30 2024-04-25T11:00:00.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.253+0000 7f3a3bb3f640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f3a340d35a0 con 0x7f3a341644e0 2024-04-25T11:00:00.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.253+0000 7f3a3bb3f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3a3406e8d0 con 0x7f3a340b8ca0 2024-04-25T11:00:00.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.253+0000 7f3a398db640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a340b8ca0 0x7f3a34163fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:00.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.253+0000 7f3a390da640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3a341644e0 0x7f3a341648f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:00.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.253+0000 7f3a3a0dc640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f3a340d1f30 0x7f3a340d2340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.69:59756/0 (socket says 172.21.15.69:59756) 2024-04-25T11:00:00.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.253+0000 7f3a3a0dc640 1 -- 172.21.15.69:0/2860430030 learned_addr learned my addr 172.21.15.69:0/2860430030 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:00.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.253+0000 7f3a398db640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a340b8ca0 0x7f3a34163fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.69:38142/0 (socket says 172.21.15.69:38142) 2024-04-25T11:00:00.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.253+0000 7f3a390da640 1 -- 172.21.15.69:0/2860430030 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a340b8ca0 msgr2=0x7f3a34163fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:00.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.253+0000 7f3a390da640 1 --2- 172.21.15.69:0/2860430030 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a340b8ca0 0x7f3a34163fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:00.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.253+0000 7f3a390da640 1 -- 172.21.15.69:0/2860430030 >> v1:172.21.15.146:6789/0 conn(0x7f3a340d1f30 legacy=0x7f3a340d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:00.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.253+0000 7f3a390da640 1 -- 172.21.15.69:0/2860430030 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a341608b0 con 0x7f3a341644e0 2024-04-25T11:00:00.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.253+0000 7f3a388d9640 1 -- 172.21.15.69:0/2860430030 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3568429421 0 0) 0x7f3a3406ed60 con 0x7f3a340d1f30 2024-04-25T11:00:00.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.253+0000 7f3a398db640 1 --2- 172.21.15.69:0/2860430030 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a340b8ca0 0x7f3a34163fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:00:00.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.254+0000 7f3a390da640 1 --2- 172.21.15.69:0/2860430030 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a341644e0 0x7f3a341648f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3a240097a0 tx=0x7f3a2402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80d71949d0203b72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:00.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.254+0000 7f3a388d9640 1 -- 172.21.15.69:0/2860430030 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a24037070 con 0x7f3a341644e0 2024-04-25T11:00:00.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.254+0000 7f3a388d9640 1 -- 172.21.15.69:0/2860430030 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a24032440 con 0x7f3a341644e0 2024-04-25T11:00:00.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.254+0000 7f3a388d9640 1 -- 172.21.15.69:0/2860430030 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a24032810 con 0x7f3a341644e0 2024-04-25T11:00:00.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.255+0000 7f3a3bb3f640 1 -- 172.21.15.69:0/2860430030 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a341644e0 msgr2=0x7f3a341648f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:00.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.255+0000 7f3a3bb3f640 1 --2- 172.21.15.69:0/2860430030 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a341644e0 0x7f3a341648f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3a240097a0 tx=0x7f3a2402a180 comp rx=0 tx=0).stop 2024-04-25T11:00:00.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.255+0000 7f3a3bb3f640 1 -- 172.21.15.69:0/2860430030 shutdown_connections 2024-04-25T11:00:00.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.255+0000 7f3a3bb3f640 1 --2- 172.21.15.69:0/2860430030 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a341644e0 0x7f3a341648f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:00.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.255+0000 7f3a3bb3f640 1 --2- 172.21.15.69:0/2860430030 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a340b8ca0 0x7f3a34163fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:00.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.255+0000 7f3a3bb3f640 1 -- 172.21.15.69:0/2860430030 >> 172.21.15.69:0/2860430030 conn(0x7f3a340d3aa0 msgr2=0x7f3a340d3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:00.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.255+0000 7f3a3bb3f640 1 -- 172.21.15.69:0/2860430030 shutdown_connections 2024-04-25T11:00:00.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.255+0000 7f3a3bb3f640 1 -- 172.21.15.69:0/2860430030 wait complete. 2024-04-25T11:00:00.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.257+0000 7f3a3bb3f640 1 Processor -- start 2024-04-25T11:00:00.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.257+0000 7f3a3bb3f640 1 -- start start 2024-04-25T11:00:00.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.258+0000 7f3a3bb3f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a340b8ca0 0x7f3a340ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:00.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.258+0000 7f3a3bb3f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a340d1f30 0x7f3a3415ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:00.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.258+0000 7f3a3bb3f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a341644e0 0x7f3a341e9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:00.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.258+0000 7f3a3bb3f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3406e280 con 0x7f3a341644e0 2024-04-25T11:00:00.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.258+0000 7f3a3bb3f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3a340d32c0 con 0x7f3a340d1f30 2024-04-25T11:00:00.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.258+0000 7f3a3bb3f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3a3406e560 con 0x7f3a340b8ca0 2024-04-25T11:00:00.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.258+0000 7f3a3a0dc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a341644e0 0x7f3a341e9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:00.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.258+0000 7f3a3a0dc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a341644e0 0x7f3a341e9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48148/0 (socket says 172.21.15.69:48148) 2024-04-25T11:00:00.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.258+0000 7f3a3a0dc640 1 -- 172.21.15.69:0/215647583 learned_addr learned my addr 172.21.15.69:0/215647583 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:00.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.258+0000 7f3a398db640 1 --2- 172.21.15.69:0/215647583 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a340b8ca0 0x7f3a340ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:00.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.258+0000 7f3a390da640 1 --2- 172.21.15.69:0/215647583 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a340d1f30 0x7f3a3415ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:00.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.259+0000 7f3a3a0dc640 1 -- 172.21.15.69:0/215647583 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a340b8ca0 msgr2=0x7f3a340ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:00.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.259+0000 7f3a3a0dc640 1 --2- 172.21.15.69:0/215647583 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a340b8ca0 0x7f3a340ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:00.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.259+0000 7f3a3a0dc640 1 -- 172.21.15.69:0/215647583 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a340d1f30 msgr2=0x7f3a3415ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:00.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.259+0000 7f3a3a0dc640 1 --2- 172.21.15.69:0/215647583 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a340d1f30 0x7f3a3415ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:00.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.259+0000 7f3a3a0dc640 1 -- 172.21.15.69:0/215647583 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a340cd3b0 con 0x7f3a341644e0 2024-04-25T11:00:00.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.259+0000 7f3a3a0dc640 1 --2- 172.21.15.69:0/215647583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a341644e0 0x7f3a341e9dc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c00c6d0 tx=0x7f3a2c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:00.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.259+0000 7f3a2a7fc640 1 -- 172.21.15.69:0/215647583 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c018070 con 0x7f3a341644e0 2024-04-25T11:00:00.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.259+0000 7f3a3bb3f640 1 -- 172.21.15.69:0/215647583 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a34161380 con 0x7f3a341644e0 2024-04-25T11:00:00.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.259+0000 7f3a2a7fc640 1 -- 172.21.15.69:0/215647583 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a2c004d60 con 0x7f3a341644e0 2024-04-25T11:00:00.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.259+0000 7f3a2a7fc640 1 -- 172.21.15.69:0/215647583 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c018070 con 0x7f3a341644e0 2024-04-25T11:00:00.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.259+0000 7f3a3bb3f640 1 -- 172.21.15.69:0/215647583 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a3415b2a0 con 0x7f3a341644e0 2024-04-25T11:00:00.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.260+0000 7f3a3bb3f640 1 -- 172.21.15.69:0/215647583 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a34050740 con 0x7f3a341644e0 2024-04-25T11:00:00.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.261+0000 7f3a2a7fc640 1 -- 172.21.15.69:0/215647583 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3a2c00a850 con 0x7f3a341644e0 2024-04-25T11:00:00.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.262+0000 7f3a2a7fc640 1 --2- 172.21.15.69:0/215647583 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3a180748e0 0x7f3a18076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:00.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.262+0000 7f3a2a7fc640 1 -- 172.21.15.69:0/215647583 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 4943+0+0 (secure 0 0 0) 0x7f3a2c004b60 con 0x7f3a341644e0 2024-04-25T11:00:00.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.262+0000 7f3a398db640 1 --2- 172.21.15.69:0/215647583 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3a180748e0 0x7f3a18076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:00.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.262+0000 7f3a398db640 1 --2- 172.21.15.69:0/215647583 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3a180748e0 0x7f3a18076df0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3a30009560 tx=0x7f3a30009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:00.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.265+0000 7f3a2a7fc640 1 -- 172.21.15.69:0/215647583 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3a2c021020 con 0x7f3a341644e0 2024-04-25T11:00:00.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:00.459+0000 7f3a3bb3f640 1 -- 172.21.15.69:0/215647583 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f3a340cd140 con 0x7f3a341644e0 2024-04-25T11:00:01.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.068+0000 7f3a2a7fc640 1 -- 172.21.15.69:0/215647583 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v25) ==== 97+0+0 (secure 0 0 0) 0x7f3a2c021020 con 0x7f3a341644e0 2024-04-25T11:00:01.067 INFO:teuthology.orchestra.run.smithi069.stderr: removed all blocklist entries 2024-04-25T11:00:01.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.070+0000 7f3a3bb3f640 1 -- 172.21.15.69:0/215647583 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3a180748e0 msgr2=0x7f3a18076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:01.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.070+0000 7f3a3bb3f640 1 --2- 172.21.15.69:0/215647583 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3a180748e0 0x7f3a18076df0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3a30009560 tx=0x7f3a30009190 comp rx=0 tx=0).stop 2024-04-25T11:00:01.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.070+0000 7f3a3bb3f640 1 -- 172.21.15.69:0/215647583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a341644e0 msgr2=0x7f3a341e9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:01.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.070+0000 7f3a3bb3f640 1 --2- 172.21.15.69:0/215647583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a341644e0 0x7f3a341e9dc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c00c6d0 tx=0x7f3a2c00cb90 comp rx=0 tx=0).stop 2024-04-25T11:00:01.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.070+0000 7f3a3bb3f640 1 -- 172.21.15.69:0/215647583 shutdown_connections 2024-04-25T11:00:01.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.070+0000 7f3a3bb3f640 1 --2- 172.21.15.69:0/215647583 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3a180748e0 0x7f3a18076df0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:01.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.070+0000 7f3a3bb3f640 1 --2- 172.21.15.69:0/215647583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a341644e0 0x7f3a341e9dc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:01.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.070+0000 7f3a3bb3f640 1 --2- 172.21.15.69:0/215647583 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a340d1f30 0x7f3a3415ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:01.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.070+0000 7f3a3bb3f640 1 --2- 172.21.15.69:0/215647583 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a340b8ca0 0x7f3a340ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:01.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.070+0000 7f3a3bb3f640 1 -- 172.21.15.69:0/215647583 >> 172.21.15.69:0/215647583 conn(0x7f3a340d3aa0 msgr2=0x7f3a3415ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:01.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.070+0000 7f3a3bb3f640 1 -- 172.21.15.69:0/215647583 shutdown_connections 2024-04-25T11:00:01.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.070+0000 7f3a3bb3f640 1 -- 172.21.15.69:0/215647583 wait complete. 2024-04-25T11:00:01.081 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T11:00:01.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.190+0000 7feea5782640 1 Processor -- start 2024-04-25T11:00:01.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.190+0000 7feea5782640 1 -- start start 2024-04-25T11:00:01.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.191+0000 7feea5782640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7feea00b3f80 0x7feea00d1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:01.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.191+0000 7feea5782640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7feea00d2240 0x7feea00b2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:01.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.191+0000 7feea5782640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feea00b4390 con 0x7feea0160870 2024-04-25T11:00:01.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.191+0000 7feea5782640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7feea00d00f0 con 0x7feea00b3f80 2024-04-25T11:00:01.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.191+0000 7feea5782640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7feea00b2470 con 0x7feea00d2240 2024-04-25T11:00:01.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.191+0000 7fee97fff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7feea00b3f80 0x7feea00d1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:01.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.191+0000 7fee97fff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7feea00b3f80 0x7feea00d1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48158/0 (socket says 172.21.15.69:48158) 2024-04-25T11:00:01.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.191+0000 7fee97fff640 1 -- 172.21.15.69:0/1144949871 learned_addr learned my addr 172.21.15.69:0/1144949871 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:01.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.191+0000 7fee9effd640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7feea0160870 0x7feea00cf300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:34782/0 (socket says 172.21.15.69:34782) 2024-04-25T11:00:01.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.192+0000 7fee97fff640 1 -- 172.21.15.69:0/1144949871 >> v2:172.21.15.146:3300/0 conn(0x7feea00d2240 msgr2=0x7feea00b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:01.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.192+0000 7fee9f7fe640 1 --2- 172.21.15.69:0/1144949871 >> v2:172.21.15.146:3300/0 conn(0x7feea00d2240 0x7feea00b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:01.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.192+0000 7fee9e7fc640 1 -- 172.21.15.69:0/1144949871 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 676799097 0 0) 0x7fee90001110 con 0x7feea0160870 2024-04-25T11:00:01.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.192+0000 7fee97fff640 1 --2- 172.21.15.69:0/1144949871 >> v2:172.21.15.146:3300/0 conn(0x7feea00d2240 0x7feea00b2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:01.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.192+0000 7fee97fff640 1 -- 172.21.15.69:0/1144949871 >> v1:172.21.15.69:6789/0 conn(0x7feea0160870 legacy=0x7feea00cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:01.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.192+0000 7fee97fff640 1 -- 172.21.15.69:0/1144949871 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea00cfb30 con 0x7feea00b3f80 2024-04-25T11:00:01.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.192+0000 7fee97fff640 1 --2- 172.21.15.69:0/1144949871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feea00b3f80 0x7feea00d1d00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7feea00c64e0 tx=0x7fee8802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832128595f3d86c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:01.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.193+0000 7fee9e7fc640 1 -- 172.21.15.69:0/1144949871 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee88037070 con 0x7feea00b3f80 2024-04-25T11:00:01.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.193+0000 7fee9e7fc640 1 -- 172.21.15.69:0/1144949871 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee88032440 con 0x7feea00b3f80 2024-04-25T11:00:01.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.193+0000 7fee9e7fc640 1 -- 172.21.15.69:0/1144949871 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee88032810 con 0x7feea00b3f80 2024-04-25T11:00:01.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.193+0000 7feea5782640 1 -- 172.21.15.69:0/1144949871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feea00b3f80 msgr2=0x7feea00d1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:01.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.193+0000 7feea5782640 1 --2- 172.21.15.69:0/1144949871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feea00b3f80 0x7feea00d1d00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7feea00c64e0 tx=0x7fee8802a270 comp rx=0 tx=0).stop 2024-04-25T11:00:01.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.193+0000 7feea5782640 1 -- 172.21.15.69:0/1144949871 shutdown_connections 2024-04-25T11:00:01.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.193+0000 7feea5782640 1 --2- 172.21.15.69:0/1144949871 >> v2:172.21.15.146:3300/0 conn(0x7feea00d2240 0x7feea00b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:01.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.193+0000 7feea5782640 1 --2- 172.21.15.69:0/1144949871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feea00b3f80 0x7feea00d1d00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:01.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.193+0000 7feea5782640 1 -- 172.21.15.69:0/1144949871 >> 172.21.15.69:0/1144949871 conn(0x7feea009b7d0 msgr2=0x7feea009bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:01.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.193+0000 7feea5782640 1 -- 172.21.15.69:0/1144949871 shutdown_connections 2024-04-25T11:00:01.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.193+0000 7feea5782640 1 -- 172.21.15.69:0/1144949871 wait complete. 2024-04-25T11:00:01.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.195+0000 7feea5782640 1 Processor -- start 2024-04-25T11:00:01.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.195+0000 7feea5782640 1 -- start start 2024-04-25T11:00:01.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.196+0000 7feea5782640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feea00b3f80 0x7feea0156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:01.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.196+0000 7feea5782640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feea00d2240 0x7feea00c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:01.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.196+0000 7feea5782640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feea0160870 0x7feea00cd660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:01.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.196+0000 7feea5782640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7feea00d2650 con 0x7feea00d2240 2024-04-25T11:00:01.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.196+0000 7feea5782640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7feea00b2100 con 0x7feea0160870 2024-04-25T11:00:01.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.196+0000 7feea5782640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7feea00af850 con 0x7feea00b3f80 2024-04-25T11:00:01.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.196+0000 7fee97fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feea00d2240 0x7feea00c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:01.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.196+0000 7fee97fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feea00d2240 0x7feea00c9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48168/0 (socket says 172.21.15.69:48168) 2024-04-25T11:00:01.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.196+0000 7fee97fff640 1 -- 172.21.15.69:0/1809510215 learned_addr learned my addr 172.21.15.69:0/1809510215 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:01.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.197+0000 7fee9f7fe640 1 --2- 172.21.15.69:0/1809510215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feea0160870 0x7feea00cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:01.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.197+0000 7fee9effd640 1 --2- 172.21.15.69:0/1809510215 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feea00b3f80 0x7feea0156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:01.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.197+0000 7fee97fff640 1 -- 172.21.15.69:0/1809510215 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feea00b3f80 msgr2=0x7feea0156040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:01.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.197+0000 7fee97fff640 1 --2- 172.21.15.69:0/1809510215 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feea00b3f80 0x7feea0156040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:01.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.197+0000 7fee97fff640 1 -- 172.21.15.69:0/1809510215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feea0160870 msgr2=0x7feea00cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:01.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.197+0000 7fee97fff640 1 --2- 172.21.15.69:0/1809510215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feea0160870 0x7feea00cd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:01.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.197+0000 7fee97fff640 1 -- 172.21.15.69:0/1809510215 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea00c9f90 con 0x7feea00d2240 2024-04-25T11:00:01.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.197+0000 7fee97fff640 1 --2- 172.21.15.69:0/1809510215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feea00d2240 0x7feea00c9a50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fee88000c00 tx=0x7fee880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:01.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.197+0000 7fee977fe640 1 -- 172.21.15.69:0/1809510215 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee88041020 con 0x7feea00d2240 2024-04-25T11:00:01.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.197+0000 7fee977fe640 1 -- 172.21.15.69:0/1809510215 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee88030970 con 0x7feea00d2240 2024-04-25T11:00:01.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.197+0000 7feea5782640 1 -- 172.21.15.69:0/1809510215 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feea00ca410 con 0x7feea00d2240 2024-04-25T11:00:01.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.197+0000 7fee977fe640 1 -- 172.21.15.69:0/1809510215 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee88037040 con 0x7feea00d2240 2024-04-25T11:00:01.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.197+0000 7feea5782640 1 -- 172.21.15.69:0/1809510215 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feea00ca9d0 con 0x7feea00d2240 2024-04-25T11:00:01.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.198+0000 7feea5782640 1 -- 172.21.15.69:0/1809510215 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feea00ca160 con 0x7feea00d2240 2024-04-25T11:00:01.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.199+0000 7fee977fe640 1 -- 172.21.15.69:0/1809510215 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fee88044020 con 0x7feea00d2240 2024-04-25T11:00:01.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.199+0000 7fee977fe640 1 --2- 172.21.15.69:0/1809510215 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fee800744b0 0x7fee800769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:01.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.199+0000 7fee977fe640 1 -- 172.21.15.69:0/1809510215 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7fee880b5880 con 0x7feea00d2240 2024-04-25T11:00:01.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.199+0000 7fee9effd640 1 --2- 172.21.15.69:0/1809510215 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fee800744b0 0x7fee800769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:01.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.200+0000 7fee9effd640 1 --2- 172.21.15.69:0/1809510215 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fee800744b0 0x7fee800769c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fee90000f60 tx=0x7fee90007c00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:01.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.204+0000 7fee977fe640 1 -- 172.21.15.69:0/1809510215 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fee8804b030 con 0x7feea00d2240 2024-04-25T11:00:01.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.450+0000 7feea5782640 1 -- 172.21.15.69:0/1809510215 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7feea0050740 con 0x7feea00d2240 2024-04-25T11:00:01.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.451+0000 7fee977fe640 1 -- 172.21.15.69:0/1809510215 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7fee880803f0 con 0x7feea00d2240 2024-04-25T11:00:01.451 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:00:01.452 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T11:00:01.452 INFO:teuthology.orchestra.run.smithi069.stdout: "auth_dump": [ 2024-04-25T11:00:01.452 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:00:01.452 INFO:teuthology.orchestra.run.smithi069.stdout: "entity": "mds.a", 2024-04-25T11:00:01.452 INFO:teuthology.orchestra.run.smithi069.stdout: "key": "AQBMNypmnESqHxAAb0BNMP4l0t7vd3uAlQOxig==", 2024-04-25T11:00:01.452 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": { 2024-04-25T11:00:01.452 INFO:teuthology.orchestra.run.smithi069.stdout: "mds": "allow", 2024-04-25T11:00:01.452 INFO:teuthology.orchestra.run.smithi069.stdout: "mgr": "allow *", 2024-04-25T11:00:01.452 INFO:teuthology.orchestra.run.smithi069.stdout: "mon": "allow *", 2024-04-25T11:00:01.452 INFO:teuthology.orchestra.run.smithi069.stdout: "osd": "allow *" 2024-04-25T11:00:01.452 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:00:01.452 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:00:01.452 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:00:01.452 INFO:teuthology.orchestra.run.smithi069.stdout: "entity": "mds.b", 2024-04-25T11:00:01.453 INFO:teuthology.orchestra.run.smithi069.stdout: "key": "AQBMNypmRLQkLBAAzJk7Zak+aQP4Xi6YayKdUw==", 2024-04-25T11:00:01.453 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": { 2024-04-25T11:00:01.453 INFO:teuthology.orchestra.run.smithi069.stdout: "mds": "allow", 2024-04-25T11:00:01.453 INFO:teuthology.orchestra.run.smithi069.stdout: "mgr": "allow *", 2024-04-25T11:00:01.453 INFO:teuthology.orchestra.run.smithi069.stdout: "mon": "allow *", 2024-04-25T11:00:01.453 INFO:teuthology.orchestra.run.smithi069.stdout: "osd": "allow *" 2024-04-25T11:00:01.453 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:00:01.453 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:00:01.453 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:00:01.453 INFO:teuthology.orchestra.run.smithi069.stdout: "entity": "mds.c", 2024-04-25T11:00:01.453 INFO:teuthology.orchestra.run.smithi069.stdout: "key": "AQBMNypmtOQPJxAAkoL9qh4tJqPgMUnKIkCoMQ==", 2024-04-25T11:00:01.453 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": { 2024-04-25T11:00:01.453 INFO:teuthology.orchestra.run.smithi069.stdout: "mds": "allow", 2024-04-25T11:00:01.454 INFO:teuthology.orchestra.run.smithi069.stdout: "mgr": "allow *", 2024-04-25T11:00:01.454 INFO:teuthology.orchestra.run.smithi069.stdout: "mon": "allow *", 2024-04-25T11:00:01.454 INFO:teuthology.orchestra.run.smithi069.stdout: "osd": "allow *" 2024-04-25T11:00:01.454 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:00:01.454 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:00:01.454 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:00:01.454 INFO:teuthology.orchestra.run.smithi069.stdout: "entity": "mds.d", 2024-04-25T11:00:01.454 INFO:teuthology.orchestra.run.smithi069.stdout: "key": "AQBMNypmI/6FMxAAyVFxJEwF2XnZ1Nb7MeIYGQ==", 2024-04-25T11:00:01.454 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": { 2024-04-25T11:00:01.454 INFO:teuthology.orchestra.run.smithi069.stdout: "mds": "allow", 2024-04-25T11:00:01.454 INFO:teuthology.orchestra.run.smithi069.stdout: "mgr": "allow *", 2024-04-25T11:00:01.454 INFO:teuthology.orchestra.run.smithi069.stdout: "mon": "allow *", 2024-04-25T11:00:01.454 INFO:teuthology.orchestra.run.smithi069.stdout: "osd": "allow *" 2024-04-25T11:00:01.454 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:00:01.455 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:00:01.455 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:00:01.455 INFO:teuthology.orchestra.run.smithi069.stdout: "entity": "osd.0", 2024-04-25T11:00:01.455 INFO:teuthology.orchestra.run.smithi069.stdout: "key": "AQBQNypmfvAiERAAoPcVgX0yuZPAEj6HhgxJ1g==", 2024-04-25T11:00:01.455 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": { 2024-04-25T11:00:01.455 INFO:teuthology.orchestra.run.smithi069.stdout: "mgr": "allow profile osd", 2024-04-25T11:00:01.455 INFO:teuthology.orchestra.run.smithi069.stdout: "mon": "allow profile osd", 2024-04-25T11:00:01.455 INFO:teuthology.orchestra.run.smithi069.stdout: "osd": "allow *" 2024-04-25T11:00:01.455 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:00:01.455 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:00:01.455 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:00:01.455 INFO:teuthology.orchestra.run.smithi069.stdout: "entity": "osd.1", 2024-04-25T11:00:01.455 INFO:teuthology.orchestra.run.smithi069.stdout: "key": "AQBTNypm1C9kEBAACwOq6Mmuxiw2Jw1tax/Sbw==", 2024-04-25T11:00:01.455 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": { 2024-04-25T11:00:01.456 INFO:teuthology.orchestra.run.smithi069.stdout: "mgr": "allow profile osd", 2024-04-25T11:00:01.456 INFO:teuthology.orchestra.run.smithi069.stdout: "mon": "allow profile osd", 2024-04-25T11:00:01.456 INFO:teuthology.orchestra.run.smithi069.stdout: "osd": "allow *" 2024-04-25T11:00:01.456 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:00:01.456 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:00:01.456 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:00:01.456 INFO:teuthology.orchestra.run.smithi069.stdout: "entity": "osd.2", 2024-04-25T11:00:01.456 INFO:teuthology.orchestra.run.smithi069.stdout: "key": "AQBWNypm32NVDhAAdRFsWgkXHNpy9Cpl+ntR1w==", 2024-04-25T11:00:01.456 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": { 2024-04-25T11:00:01.456 INFO:teuthology.orchestra.run.smithi069.stdout: "mgr": "allow profile osd", 2024-04-25T11:00:01.456 INFO:teuthology.orchestra.run.smithi069.stdout: "mon": "allow profile osd", 2024-04-25T11:00:01.456 INFO:teuthology.orchestra.run.smithi069.stdout: "osd": "allow *" 2024-04-25T11:00:01.456 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:00:01.456 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:00:01.456 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:00:01.457 INFO:teuthology.orchestra.run.smithi069.stdout: "entity": "osd.3", 2024-04-25T11:00:01.457 INFO:teuthology.orchestra.run.smithi069.stdout: "key": "AQBZNypm7IIzDRAAfXeNYJ42bXE4TX2F8EAevA==", 2024-04-25T11:00:01.457 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": { 2024-04-25T11:00:01.457 INFO:teuthology.orchestra.run.smithi069.stdout: "mgr": "allow profile osd", 2024-04-25T11:00:01.457 INFO:teuthology.orchestra.run.smithi069.stdout: "mon": "allow profile osd", 2024-04-25T11:00:01.457 INFO:teuthology.orchestra.run.smithi069.stdout: "osd": "allow *" 2024-04-25T11:00:01.457 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:00:01.457 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:00:01.457 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:00:01.457 INFO:teuthology.orchestra.run.smithi069.stdout: "entity": "osd.4", 2024-04-25T11:00:01.457 INFO:teuthology.orchestra.run.smithi069.stdout: "key": "AQBfNypmFTGrDxAAITl0VOssjLlcOYiug6B0pA==", 2024-04-25T11:00:01.457 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": { 2024-04-25T11:00:01.457 INFO:teuthology.orchestra.run.smithi069.stdout: "mgr": "allow profile osd", 2024-04-25T11:00:01.458 INFO:teuthology.orchestra.run.smithi069.stdout: "mon": "allow profile osd", 2024-04-25T11:00:01.458 INFO:teuthology.orchestra.run.smithi069.stdout: "osd": "allow *" 2024-04-25T11:00:01.458 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:00:01.458 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:00:01.458 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:00:01.458 INFO:teuthology.orchestra.run.smithi069.stdout: "entity": "osd.5", 2024-04-25T11:00:01.458 INFO:teuthology.orchestra.run.smithi069.stdout: "key": "AQBiNypmRgHLDhAAaCiszWXZ8DDTHuapyFE3lQ==", 2024-04-25T11:00:01.458 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": { 2024-04-25T11:00:01.458 INFO:teuthology.orchestra.run.smithi069.stdout: "mgr": "allow profile osd", 2024-04-25T11:00:01.458 INFO:teuthology.orchestra.run.smithi069.stdout: "mon": "allow profile osd", 2024-04-25T11:00:01.458 INFO:teuthology.orchestra.run.smithi069.stdout: "osd": "allow *" 2024-04-25T11:00:01.458 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:00:01.458 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:00:01.458 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:00:01.459 INFO:teuthology.orchestra.run.smithi069.stdout: "entity": "osd.6", 2024-04-25T11:00:01.459 INFO:teuthology.orchestra.run.smithi069.stdout: "key": "AQBlNypme8AZDhAA1syulvLhqH27AbYYuSWnVw==", 2024-04-25T11:00:01.459 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": { 2024-04-25T11:00:01.459 INFO:teuthology.orchestra.run.smithi069.stdout: "mgr": "allow profile osd", 2024-04-25T11:00:01.459 INFO:teuthology.orchestra.run.smithi069.stdout: "mon": "allow profile osd", 2024-04-25T11:00:01.459 INFO:teuthology.orchestra.run.smithi069.stdout: "osd": "allow *" 2024-04-25T11:00:01.459 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:00:01.459 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:00:01.459 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:00:01.459 INFO:teuthology.orchestra.run.smithi069.stdout: "entity": "osd.7", 2024-04-25T11:00:01.459 INFO:teuthology.orchestra.run.smithi069.stdout: "key": "AQBoNypmtrloEBAAm8UktQI0CNss2Goap/EddA==", 2024-04-25T11:00:01.459 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": { 2024-04-25T11:00:01.459 INFO:teuthology.orchestra.run.smithi069.stdout: "mgr": "allow profile osd", 2024-04-25T11:00:01.459 INFO:teuthology.orchestra.run.smithi069.stdout: "mon": "allow profile osd", 2024-04-25T11:00:01.460 INFO:teuthology.orchestra.run.smithi069.stdout: "osd": "allow *" 2024-04-25T11:00:01.460 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:00:01.460 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:00:01.460 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:00:01.460 INFO:teuthology.orchestra.run.smithi069.stdout: "entity": "client.0", 2024-04-25T11:00:01.460 INFO:teuthology.orchestra.run.smithi069.stdout: "key": "AQBMNypmtv/zNxAAz2JcqUNyHu2ipkrbl8ONng==", 2024-04-25T11:00:01.460 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": { 2024-04-25T11:00:01.460 INFO:teuthology.orchestra.run.smithi069.stdout: "mds": "allow", 2024-04-25T11:00:01.460 INFO:teuthology.orchestra.run.smithi069.stdout: "mgr": "allow r", 2024-04-25T11:00:01.460 INFO:teuthology.orchestra.run.smithi069.stdout: "mon": "allow rw", 2024-04-25T11:00:01.460 INFO:teuthology.orchestra.run.smithi069.stdout: "osd": "allow rwx" 2024-04-25T11:00:01.460 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:00:01.460 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:00:01.460 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:00:01.460 INFO:teuthology.orchestra.run.smithi069.stdout: "entity": "client.1", 2024-04-25T11:00:01.461 INFO:teuthology.orchestra.run.smithi069.stdout: "key": "AQBNNypm5U2DAxAAG+NYDPdMtwS4CvAtln+8Aw==", 2024-04-25T11:00:01.461 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": { 2024-04-25T11:00:01.461 INFO:teuthology.orchestra.run.smithi069.stdout: "mds": "allow", 2024-04-25T11:00:01.461 INFO:teuthology.orchestra.run.smithi069.stdout: "mgr": "allow r", 2024-04-25T11:00:01.461 INFO:teuthology.orchestra.run.smithi069.stdout: "mon": "allow rw", 2024-04-25T11:00:01.461 INFO:teuthology.orchestra.run.smithi069.stdout: "osd": "allow rwx" 2024-04-25T11:00:01.461 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:00:01.461 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:00:01.461 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:00:01.461 INFO:teuthology.orchestra.run.smithi069.stdout: "entity": "client.2", 2024-04-25T11:00:01.461 INFO:teuthology.orchestra.run.smithi069.stdout: "key": "AQBMNypmTQeEOxAAqM/YzZlM2cz7zo2g78NCMw==", 2024-04-25T11:00:01.461 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": { 2024-04-25T11:00:01.461 INFO:teuthology.orchestra.run.smithi069.stdout: "mds": "allow", 2024-04-25T11:00:01.461 INFO:teuthology.orchestra.run.smithi069.stdout: "mgr": "allow r", 2024-04-25T11:00:01.461 INFO:teuthology.orchestra.run.smithi069.stdout: "mon": "allow rw", 2024-04-25T11:00:01.462 INFO:teuthology.orchestra.run.smithi069.stdout: "osd": "allow rwx" 2024-04-25T11:00:01.462 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:00:01.462 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:00:01.462 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:00:01.462 INFO:teuthology.orchestra.run.smithi069.stdout: "entity": "client.3", 2024-04-25T11:00:01.462 INFO:teuthology.orchestra.run.smithi069.stdout: "key": "AQBNNypm7oIZBxAAABnro3onV9BFztCKMJapkA==", 2024-04-25T11:00:01.462 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": { 2024-04-25T11:00:01.462 INFO:teuthology.orchestra.run.smithi069.stdout: "mds": "allow", 2024-04-25T11:00:01.462 INFO:teuthology.orchestra.run.smithi069.stdout: "mgr": "allow r", 2024-04-25T11:00:01.462 INFO:teuthology.orchestra.run.smithi069.stdout: "mon": "allow rw", 2024-04-25T11:00:01.462 INFO:teuthology.orchestra.run.smithi069.stdout: "osd": "allow rwx" 2024-04-25T11:00:01.462 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:00:01.462 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:00:01.463 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:00:01.463 INFO:teuthology.orchestra.run.smithi069.stdout: "entity": "client.admin", 2024-04-25T11:00:01.463 INFO:teuthology.orchestra.run.smithi069.stdout: "key": "AQBMNypmZcsDAxAASE3yShmjBL0EPbczE92psg==", 2024-04-25T11:00:01.463 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": { 2024-04-25T11:00:01.463 INFO:teuthology.orchestra.run.smithi069.stdout: "mds": "allow *", 2024-04-25T11:00:01.463 INFO:teuthology.orchestra.run.smithi069.stdout: "mgr": "allow *", 2024-04-25T11:00:01.463 INFO:teuthology.orchestra.run.smithi069.stdout: "mon": "allow *", 2024-04-25T11:00:01.463 INFO:teuthology.orchestra.run.smithi069.stdout: "osd": "allow *" 2024-04-25T11:00:01.463 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:00:01.463 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:00:01.463 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:00:01.463 INFO:teuthology.orchestra.run.smithi069.stdout: "entity": "client.bootstrap-mds", 2024-04-25T11:00:01.463 INFO:teuthology.orchestra.run.smithi069.stdout: "key": "AQBuNypmSpniEBAAcB53H1hHxRdnIcBHHpoyxg==", 2024-04-25T11:00:01.463 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": { 2024-04-25T11:00:01.463 INFO:teuthology.orchestra.run.smithi069.stdout: "mon": "allow profile bootstrap-mds" 2024-04-25T11:00:01.464 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:00:01.464 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:00:01.464 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:00:01.464 INFO:teuthology.orchestra.run.smithi069.stdout: "entity": "client.bootstrap-mgr", 2024-04-25T11:00:01.464 INFO:teuthology.orchestra.run.smithi069.stdout: "key": "AQBuNypmSqHiEBAA9uOX8seru1bUPRC87ofzlw==", 2024-04-25T11:00:01.464 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": { 2024-04-25T11:00:01.464 INFO:teuthology.orchestra.run.smithi069.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-25T11:00:01.464 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:00:01.464 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:00:01.464 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:00:01.464 INFO:teuthology.orchestra.run.smithi069.stdout: "entity": "client.bootstrap-osd", 2024-04-25T11:00:01.464 INFO:teuthology.orchestra.run.smithi069.stdout: "key": "AQBuNypm+afiEBAA1C6VBu8TDPovK2q4+9DbVQ==", 2024-04-25T11:00:01.464 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": { 2024-04-25T11:00:01.464 INFO:teuthology.orchestra.run.smithi069.stdout: "mon": "allow profile bootstrap-osd" 2024-04-25T11:00:01.465 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:00:01.465 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:00:01.465 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:00:01.465 INFO:teuthology.orchestra.run.smithi069.stdout: "entity": "client.bootstrap-rbd", 2024-04-25T11:00:01.465 INFO:teuthology.orchestra.run.smithi069.stdout: "key": "AQBuNypms67iEBAAK6aCIRjm5IrpeZx5LHm51A==", 2024-04-25T11:00:01.465 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": { 2024-04-25T11:00:01.465 INFO:teuthology.orchestra.run.smithi069.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-25T11:00:01.465 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:00:01.465 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:00:01.465 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:00:01.465 INFO:teuthology.orchestra.run.smithi069.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-25T11:00:01.465 INFO:teuthology.orchestra.run.smithi069.stdout: "key": "AQBuNypmm7fiEBAA4XzmOV3phrlqvUf0k9a6IA==", 2024-04-25T11:00:01.465 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": { 2024-04-25T11:00:01.465 INFO:teuthology.orchestra.run.smithi069.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-25T11:00:01.466 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:00:01.466 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:00:01.466 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:00:01.466 INFO:teuthology.orchestra.run.smithi069.stdout: "entity": "client.bootstrap-rgw", 2024-04-25T11:00:01.466 INFO:teuthology.orchestra.run.smithi069.stdout: "key": "AQBuNypmxb7iEBAAQlyLflKQ2hk93T8BCdyMvQ==", 2024-04-25T11:00:01.466 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": { 2024-04-25T11:00:01.466 INFO:teuthology.orchestra.run.smithi069.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-25T11:00:01.466 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:00:01.466 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:00:01.466 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:00:01.466 INFO:teuthology.orchestra.run.smithi069.stdout: "entity": "mgr.x", 2024-04-25T11:00:01.466 INFO:teuthology.orchestra.run.smithi069.stdout: "key": "AQBMNypmq1MRHBAACDY/Y25XF2oiwMi3/qR7wQ==", 2024-04-25T11:00:01.466 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": { 2024-04-25T11:00:01.466 INFO:teuthology.orchestra.run.smithi069.stdout: "mds": "allow *", 2024-04-25T11:00:01.466 INFO:teuthology.orchestra.run.smithi069.stdout: "mon": "allow profile mgr", 2024-04-25T11:00:01.467 INFO:teuthology.orchestra.run.smithi069.stdout: "osd": "allow *" 2024-04-25T11:00:01.467 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:00:01.467 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:00:01.467 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:00:01.467 INFO:teuthology.orchestra.run.smithi069.stdout: "entity": "mgr.y", 2024-04-25T11:00:01.467 INFO:teuthology.orchestra.run.smithi069.stdout: "key": "AQBMNypm4Gx8FxAAsS/vgwzV6B3i6oVbp+gRqQ==", 2024-04-25T11:00:01.467 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": { 2024-04-25T11:00:01.467 INFO:teuthology.orchestra.run.smithi069.stdout: "mds": "allow *", 2024-04-25T11:00:01.467 INFO:teuthology.orchestra.run.smithi069.stdout: "mon": "allow profile mgr", 2024-04-25T11:00:01.467 INFO:teuthology.orchestra.run.smithi069.stdout: "osd": "allow *" 2024-04-25T11:00:01.467 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:00:01.467 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:00:01.467 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-04-25T11:00:01.467 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T11:00:01.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.453+0000 7feea5782640 1 -- 172.21.15.69:0/1809510215 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fee800744b0 msgr2=0x7fee800769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:01.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.453+0000 7feea5782640 1 --2- 172.21.15.69:0/1809510215 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fee800744b0 0x7fee800769c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fee90000f60 tx=0x7fee90007c00 comp rx=0 tx=0).stop 2024-04-25T11:00:01.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.453+0000 7feea5782640 1 -- 172.21.15.69:0/1809510215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feea00d2240 msgr2=0x7feea00c9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:01.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.453+0000 7feea5782640 1 --2- 172.21.15.69:0/1809510215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feea00d2240 0x7feea00c9a50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fee88000c00 tx=0x7fee880046c0 comp rx=0 tx=0).stop 2024-04-25T11:00:01.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.453+0000 7feea5782640 1 -- 172.21.15.69:0/1809510215 shutdown_connections 2024-04-25T11:00:01.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.453+0000 7feea5782640 1 --2- 172.21.15.69:0/1809510215 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fee800744b0 0x7fee800769c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:01.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.453+0000 7feea5782640 1 --2- 172.21.15.69:0/1809510215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feea0160870 0x7feea00cd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:01.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.453+0000 7feea5782640 1 --2- 172.21.15.69:0/1809510215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feea00d2240 0x7feea00c9a50 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:01.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.453+0000 7feea5782640 1 --2- 172.21.15.69:0/1809510215 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feea00b3f80 0x7feea0156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:01.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.453+0000 7feea5782640 1 -- 172.21.15.69:0/1809510215 >> 172.21.15.69:0/1809510215 conn(0x7feea009b7d0 msgr2=0x7feea0071910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:01.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.453+0000 7feea5782640 1 -- 172.21.15.69:0/1809510215 shutdown_connections 2024-04-25T11:00:01.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.453+0000 7feea5782640 1 -- 172.21.15.69:0/1809510215 wait complete. 2024-04-25T11:00:01.470 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-25T11:00:01.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.577+0000 7f460a4af640 1 Processor -- start 2024-04-25T11:00:01.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.577+0000 7f460a4af640 1 -- start start 2024-04-25T11:00:01.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.578+0000 7f460a4af640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f46040b91d0 0x7f4604161e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:01.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.578+0000 7f460a4af640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f46040cf770 0x7f46040b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:01.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.578+0000 7f460a4af640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46040cfc30 con 0x7f46041623c0 2024-04-25T11:00:01.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.578+0000 7f460a4af640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f46040d1ff0 con 0x7f46040b91d0 2024-04-25T11:00:01.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.578+0000 7f460a4af640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f46040d2890 con 0x7f46040cf770 2024-04-25T11:00:01.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.578+0000 7f4603fff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f46040b91d0 0x7f4604161e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:01.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.578+0000 7f4603fff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f46040b91d0 0x7f4604161e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48176/0 (socket says 172.21.15.69:48176) 2024-04-25T11:00:01.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.578+0000 7f46037fe640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f46041623c0 0x7f46041627d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:34794/0 (socket says 172.21.15.69:34794) 2024-04-25T11:00:01.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.578+0000 7f4603fff640 1 -- 172.21.15.69:0/1940747268 learned_addr learned my addr 172.21.15.69:0/1940747268 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:01.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.579+0000 7f4608a4c640 1 --2- 172.21.15.69:0/1940747268 >> v2:172.21.15.146:3300/0 conn(0x7f46040cf770 0x7f46040b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:01.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.579+0000 7f4603fff640 1 -- 172.21.15.69:0/1940747268 >> v2:172.21.15.146:3300/0 conn(0x7f46040cf770 msgr2=0x7f46040b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:01.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.579+0000 7f4603fff640 1 --2- 172.21.15.69:0/1940747268 >> v2:172.21.15.146:3300/0 conn(0x7f46040cf770 0x7f46040b0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:01.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.579+0000 7f4603fff640 1 -- 172.21.15.69:0/1940747268 >> v1:172.21.15.69:6789/0 conn(0x7f46041623c0 legacy=0x7f46041627d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:01.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.579+0000 7f4602ffd640 1 -- 172.21.15.69:0/1940747268 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3362339640 0 0) 0x7f46040cfc30 con 0x7f46041623c0 2024-04-25T11:00:01.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.579+0000 7f4603fff640 1 -- 172.21.15.69:0/1940747268 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46040d2240 con 0x7f46040b91d0 2024-04-25T11:00:01.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.579+0000 7f4603fff640 1 --2- 172.21.15.69:0/1940747268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46040b91d0 0x7f4604161e80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f46040caa70 tx=0x7f45f402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a9a1744eaa65d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:01.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.580+0000 7f4602ffd640 1 -- 172.21.15.69:0/1940747268 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f4037070 con 0x7f46040b91d0 2024-04-25T11:00:01.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.580+0000 7f4602ffd640 1 -- 172.21.15.69:0/1940747268 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45f4032440 con 0x7f46040b91d0 2024-04-25T11:00:01.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.580+0000 7f4602ffd640 1 -- 172.21.15.69:0/1940747268 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f4032810 con 0x7f46040b91d0 2024-04-25T11:00:01.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.580+0000 7f460a4af640 1 -- 172.21.15.69:0/1940747268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46040b91d0 msgr2=0x7f4604161e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:01.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.580+0000 7f460a4af640 1 --2- 172.21.15.69:0/1940747268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46040b91d0 0x7f4604161e80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f46040caa70 tx=0x7f45f402a1e0 comp rx=0 tx=0).stop 2024-04-25T11:00:01.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.580+0000 7f460a4af640 1 -- 172.21.15.69:0/1940747268 shutdown_connections 2024-04-25T11:00:01.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.580+0000 7f460a4af640 1 --2- 172.21.15.69:0/1940747268 >> v2:172.21.15.146:3300/0 conn(0x7f46040cf770 0x7f46040b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:01.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.580+0000 7f460a4af640 1 --2- 172.21.15.69:0/1940747268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46040b91d0 0x7f4604161e80 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:01.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.580+0000 7f460a4af640 1 -- 172.21.15.69:0/1940747268 >> 172.21.15.69:0/1940747268 conn(0x7f46040d3aa0 msgr2=0x7f46040d3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:01.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.580+0000 7f460a4af640 1 -- 172.21.15.69:0/1940747268 shutdown_connections 2024-04-25T11:00:01.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.580+0000 7f460a4af640 1 -- 172.21.15.69:0/1940747268 wait complete. 2024-04-25T11:00:01.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.582+0000 7f460a4af640 1 Processor -- start 2024-04-25T11:00:01.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.582+0000 7f460a4af640 1 -- start start 2024-04-25T11:00:01.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.583+0000 7f460a4af640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f46040b91d0 0x7f4604155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:01.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.583+0000 7f460a4af640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46041623c0 0x7f46040b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:01.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.584+0000 7f460a4af640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46041560e0 0x7f46040cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:01.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.584+0000 7f460a4af640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f46040b24a0 con 0x7f46041623c0 2024-04-25T11:00:01.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.584+0000 7f460a4af640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f46040b2320 con 0x7f46041560e0 2024-04-25T11:00:01.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.584+0000 7f460a4af640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f46040b0bd0 con 0x7f46040b91d0 2024-04-25T11:00:01.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.584+0000 7f46037fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46041623c0 0x7f46040b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:01.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.584+0000 7f46037fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46041623c0 0x7f46040b26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48178/0 (socket says 172.21.15.69:48178) 2024-04-25T11:00:01.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.584+0000 7f46037fe640 1 -- 172.21.15.69:0/3011181540 learned_addr learned my addr 172.21.15.69:0/3011181540 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:01.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.584+0000 7f4603fff640 1 --2- 172.21.15.69:0/3011181540 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f46040b91d0 0x7f4604155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:01.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.584+0000 7f4608a4c640 1 --2- 172.21.15.69:0/3011181540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46041560e0 0x7f46040cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:01.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.584+0000 7f46037fe640 1 -- 172.21.15.69:0/3011181540 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f46040b91d0 msgr2=0x7f4604155a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:01.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.584+0000 7f46037fe640 1 --2- 172.21.15.69:0/3011181540 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f46040b91d0 0x7f4604155a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:01.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.584+0000 7f46037fe640 1 -- 172.21.15.69:0/3011181540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46041560e0 msgr2=0x7f46040cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:01.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.584+0000 7f46037fe640 1 --2- 172.21.15.69:0/3011181540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46041560e0 0x7f46040cd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:01.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.584+0000 7f46037fe640 1 -- 172.21.15.69:0/3011181540 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46040b2f90 con 0x7f46041623c0 2024-04-25T11:00:01.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.584+0000 7f4608a4c640 1 --2- 172.21.15.69:0/3011181540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46041560e0 0x7f46040cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:01.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.584+0000 7f46037fe640 1 --2- 172.21.15.69:0/3011181540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46041623c0 0x7f46040b26b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f45f800c460 tx=0x7f45f800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:01.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.584+0000 7f4600ff9640 1 -- 172.21.15.69:0/3011181540 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8018070 con 0x7f46041623c0 2024-04-25T11:00:01.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.584+0000 7f4600ff9640 1 -- 172.21.15.69:0/3011181540 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45f8007500 con 0x7f46041623c0 2024-04-25T11:00:01.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.584+0000 7f460a4af640 1 -- 172.21.15.69:0/3011181540 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46040b3270 con 0x7f46041623c0 2024-04-25T11:00:01.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.584+0000 7f4600ff9640 1 -- 172.21.15.69:0/3011181540 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8004a20 con 0x7f46041623c0 2024-04-25T11:00:01.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.584+0000 7f460a4af640 1 -- 172.21.15.69:0/3011181540 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46040cac80 con 0x7f46041623c0 2024-04-25T11:00:01.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.585+0000 7f460a4af640 1 -- 172.21.15.69:0/3011181540 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4604050740 con 0x7f46041623c0 2024-04-25T11:00:01.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.586+0000 7f4600ff9640 1 -- 172.21.15.69:0/3011181540 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f45f801b070 con 0x7f46041623c0 2024-04-25T11:00:01.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.587+0000 7f4600ff9640 1 --2- 172.21.15.69:0/3011181540 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f45f00744b0 0x7f45f00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:01.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.587+0000 7f4603fff640 1 --2- 172.21.15.69:0/3011181540 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f45f00744b0 0x7f45f00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:01.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.587+0000 7f4600ff9640 1 -- 172.21.15.69:0/3011181540 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7f45f8094070 con 0x7f46041623c0 2024-04-25T11:00:01.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.587+0000 7f4603fff640 1 --2- 172.21.15.69:0/3011181540 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f45f00744b0 0x7f45f00769c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f45f4034000 tx=0x7f45f402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:01.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.591+0000 7f4600ff9640 1 -- 172.21.15.69:0/3011181540 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f45f805e590 con 0x7f46041623c0 2024-04-25T11:00:01.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.834+0000 7f460a4af640 1 -- 172.21.15.69:0/3011181540 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f46040cf970 con 0x7f46041623c0 2024-04-25T11:00:01.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.837+0000 7f4600ff9640 1 -- 172.21.15.69:0/3011181540 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f45f8062890 con 0x7f46041623c0 2024-04-25T11:00:01.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.839+0000 7f460a4af640 1 -- 172.21.15.69:0/3011181540 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f45f00744b0 msgr2=0x7f45f00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:01.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.839+0000 7f460a4af640 1 --2- 172.21.15.69:0/3011181540 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f45f00744b0 0x7f45f00769c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f45f4034000 tx=0x7f45f402e000 comp rx=0 tx=0).stop 2024-04-25T11:00:01.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.839+0000 7f460a4af640 1 -- 172.21.15.69:0/3011181540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46041623c0 msgr2=0x7f46040b26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:01.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.839+0000 7f460a4af640 1 --2- 172.21.15.69:0/3011181540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46041623c0 0x7f46040b26b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f45f800c460 tx=0x7f45f800c920 comp rx=0 tx=0).stop 2024-04-25T11:00:01.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.839+0000 7f460a4af640 1 -- 172.21.15.69:0/3011181540 shutdown_connections 2024-04-25T11:00:01.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.839+0000 7f460a4af640 1 --2- 172.21.15.69:0/3011181540 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f45f00744b0 0x7f45f00769c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:01.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.839+0000 7f460a4af640 1 --2- 172.21.15.69:0/3011181540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46041560e0 0x7f46040cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:01.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.839+0000 7f460a4af640 1 --2- 172.21.15.69:0/3011181540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46041623c0 0x7f46040b26b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:01.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.839+0000 7f460a4af640 1 --2- 172.21.15.69:0/3011181540 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f46040b91d0 0x7f4604155a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:01.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.839+0000 7f460a4af640 1 -- 172.21.15.69:0/3011181540 >> 172.21.15.69:0/3011181540 conn(0x7f46040d3aa0 msgr2=0x7f46040ce720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:01.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.839+0000 7f460a4af640 1 -- 172.21.15.69:0/3011181540 shutdown_connections 2024-04-25T11:00:01.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.839+0000 7f460a4af640 1 -- 172.21.15.69:0/3011181540 wait complete. 2024-04-25T11:00:01.850 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-25T11:00:01.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.959+0000 7f1ed3c69640 1 Processor -- start 2024-04-25T11:00:01.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.959+0000 7f1ed3c69640 1 -- start start 2024-04-25T11:00:01.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.960+0000 7f1ed3c69640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ecc0d0e00 0x7f1ecc0d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:01.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.960+0000 7f1ed3c69640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1ecc0d4a50 0x7f1ecc0d4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:01.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.960+0000 7f1ed3c69640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ecc0d1850 con 0x7f1ecc06c5c0 2024-04-25T11:00:01.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.960+0000 7f1ed3c69640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f1ecc160de0 con 0x7f1ecc0d4a50 2024-04-25T11:00:01.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.960+0000 7f1ed3c69640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1ecc1610c0 con 0x7f1ecc0d0e00 2024-04-25T11:00:01.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.960+0000 7f1ed1204640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f1ecc06c5c0 0x7f1ecc06c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:34804/0 (socket says 172.21.15.69:34804) 2024-04-25T11:00:01.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.960+0000 7f1ed1204640 1 -- 172.21.15.69:0/1370020999 learned_addr learned my addr 172.21.15.69:0/1370020999 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:01.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.960+0000 7f1ed1a05640 1 --2- 172.21.15.69:0/1370020999 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ecc0d0e00 0x7f1ecc0d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:01.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.960+0000 7f1ed2206640 1 --2- 172.21.15.69:0/1370020999 >> v2:172.21.15.146:3300/0 conn(0x7f1ecc0d4a50 0x7f1ecc0d4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:01.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.961+0000 7f1ed0a03640 1 -- 172.21.15.69:0/1370020999 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1371537752 0 0) 0x7f1ecc0d1850 con 0x7f1ecc06c5c0 2024-04-25T11:00:01.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.961+0000 7f1ed0a03640 1 -- 172.21.15.69:0/1370020999 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1eb4003520 con 0x7f1ecc06c5c0 2024-04-25T11:00:01.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.961+0000 7f1ed1a05640 1 -- 172.21.15.69:0/1370020999 >> v2:172.21.15.146:3300/0 conn(0x7f1ecc0d4a50 msgr2=0x7f1ecc0d4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:01.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.961+0000 7f1ed1a05640 1 --2- 172.21.15.69:0/1370020999 >> v2:172.21.15.146:3300/0 conn(0x7f1ecc0d4a50 0x7f1ecc0d4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:01.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.961+0000 7f1ed1a05640 1 -- 172.21.15.69:0/1370020999 >> v1:172.21.15.69:6789/0 conn(0x7f1ecc06c5c0 legacy=0x7f1ecc06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:01.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.961+0000 7f1ed1a05640 1 -- 172.21.15.69:0/1370020999 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ecc1613a0 con 0x7f1ecc0d0e00 2024-04-25T11:00:01.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.962+0000 7f1ed1a05640 1 --2- 172.21.15.69:0/1370020999 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ecc0d0e00 0x7f1ecc0d1210 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009d90 tx=0x7f1ebc02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7152071b3b5adcef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:01.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.962+0000 7f1ed0a03640 1 -- 172.21.15.69:0/1370020999 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc007740 con 0x7f1ecc0d0e00 2024-04-25T11:00:01.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.962+0000 7f1ed0a03640 1 -- 172.21.15.69:0/1370020999 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ebc0078e0 con 0x7f1ecc0d0e00 2024-04-25T11:00:01.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.962+0000 7f1ed0a03640 1 -- 172.21.15.69:0/1370020999 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc007c90 con 0x7f1ecc0d0e00 2024-04-25T11:00:01.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.963+0000 7f1ed3c69640 1 -- 172.21.15.69:0/1370020999 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ecc0d0e00 msgr2=0x7f1ecc0d1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:01.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.963+0000 7f1ed3c69640 1 --2- 172.21.15.69:0/1370020999 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ecc0d0e00 0x7f1ecc0d1210 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009d90 tx=0x7f1ebc02a660 comp rx=0 tx=0).stop 2024-04-25T11:00:01.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.963+0000 7f1ed3c69640 1 -- 172.21.15.69:0/1370020999 shutdown_connections 2024-04-25T11:00:01.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.963+0000 7f1ed3c69640 1 --2- 172.21.15.69:0/1370020999 >> v2:172.21.15.146:3300/0 conn(0x7f1ecc0d4a50 0x7f1ecc0d4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:01.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.963+0000 7f1ed3c69640 1 --2- 172.21.15.69:0/1370020999 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ecc0d0e00 0x7f1ecc0d1210 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:01.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.963+0000 7f1ed3c69640 1 -- 172.21.15.69:0/1370020999 >> 172.21.15.69:0/1370020999 conn(0x7f1ecc0d5d90 msgr2=0x7f1ecc0d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:01.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.963+0000 7f1ed3c69640 1 -- 172.21.15.69:0/1370020999 shutdown_connections 2024-04-25T11:00:01.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.963+0000 7f1ed3c69640 1 -- 172.21.15.69:0/1370020999 wait complete. 2024-04-25T11:00:01.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.964+0000 7f1ed3c69640 1 Processor -- start 2024-04-25T11:00:01.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.965+0000 7f1ed3c69640 1 -- start start 2024-04-25T11:00:01.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.966+0000 7f1ed3c69640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ecc06c5c0 0x7f1ecc1f97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:01.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.966+0000 7f1ed3c69640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ecc0d0e00 0x7f1ecc0cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:01.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.966+0000 7f1ed3c69640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ecc0d4a50 0x7f1ecc0cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:01.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.966+0000 7f1ed3c69640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc164e00 con 0x7f1ecc0d0e00 2024-04-25T11:00:01.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.966+0000 7f1ed3c69640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc0b5320 con 0x7f1ecc06c5c0 2024-04-25T11:00:01.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.966+0000 7f1ed3c69640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1ecc164f80 con 0x7f1ecc0d4a50 2024-04-25T11:00:01.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.966+0000 7f1ed1204640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ecc0d0e00 0x7f1ecc0cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:01.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.966+0000 7f1ed1204640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ecc0d0e00 0x7f1ecc0cefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48180/0 (socket says 172.21.15.69:48180) 2024-04-25T11:00:01.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.966+0000 7f1ed1204640 1 -- 172.21.15.69:0/111020076 learned_addr learned my addr 172.21.15.69:0/111020076 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:01.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.966+0000 7f1ed2206640 1 --2- 172.21.15.69:0/111020076 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ecc0d4a50 0x7f1ecc0cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:01.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.966+0000 7f1ed1a05640 1 --2- 172.21.15.69:0/111020076 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ecc06c5c0 0x7f1ecc1f97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:01.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.966+0000 7f1ed1204640 1 -- 172.21.15.69:0/111020076 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ecc0d4a50 msgr2=0x7f1ecc0cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:01.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.966+0000 7f1ed1204640 1 --2- 172.21.15.69:0/111020076 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ecc0d4a50 0x7f1ecc0cf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:01.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.966+0000 7f1ed1204640 1 -- 172.21.15.69:0/111020076 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ecc06c5c0 msgr2=0x7f1ecc1f97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:01.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.966+0000 7f1ed1204640 1 --2- 172.21.15.69:0/111020076 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ecc06c5c0 0x7f1ecc1f97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:01.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.966+0000 7f1ed1204640 1 -- 172.21.15.69:0/111020076 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ecc0cb8d0 con 0x7f1ecc0d0e00 2024-04-25T11:00:01.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.966+0000 7f1ed1a05640 1 --2- 172.21.15.69:0/111020076 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ecc06c5c0 0x7f1ecc1f97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:00:01.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.966+0000 7f1ed1204640 1 --2- 172.21.15.69:0/111020076 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ecc0d0e00 0x7f1ecc0cefa0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1ec000c410 tx=0x7f1ec000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:01.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.966+0000 7f1eca7fc640 1 -- 172.21.15.69:0/111020076 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0017070 con 0x7f1ecc0d0e00 2024-04-25T11:00:01.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.967+0000 7f1eca7fc640 1 -- 172.21.15.69:0/111020076 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ec0005ce0 con 0x7f1ecc0d0e00 2024-04-25T11:00:01.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.967+0000 7f1ed3c69640 1 -- 172.21.15.69:0/111020076 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ecc0cbbb0 con 0x7f1ecc0d0e00 2024-04-25T11:00:01.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.967+0000 7f1eca7fc640 1 -- 172.21.15.69:0/111020076 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec00072f0 con 0x7f1ecc0d0e00 2024-04-25T11:00:01.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.967+0000 7f1ed3c69640 1 -- 172.21.15.69:0/111020076 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ecc0cc170 con 0x7f1ecc0d0e00 2024-04-25T11:00:01.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.967+0000 7f1ed3c69640 1 -- 172.21.15.69:0/111020076 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ecc050740 con 0x7f1ecc0d0e00 2024-04-25T11:00:01.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.968+0000 7f1eca7fc640 1 -- 172.21.15.69:0/111020076 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1ec001a070 con 0x7f1ecc0d0e00 2024-04-25T11:00:01.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.969+0000 7f1eca7fc640 1 --2- 172.21.15.69:0/111020076 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1eb80744b0 0x7f1eb80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:01.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.969+0000 7f1ed1a05640 1 --2- 172.21.15.69:0/111020076 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1eb80744b0 0x7f1eb80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:01.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.969+0000 7f1eca7fc640 1 -- 172.21.15.69:0/111020076 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7f1ec0094360 con 0x7f1ecc0d0e00 2024-04-25T11:00:01.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.969+0000 7f1ed1a05640 1 --2- 172.21.15.69:0/111020076 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1eb80744b0 0x7f1eb80769c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc034000 tx=0x7f1ebc02f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:01.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:01.974+0000 7f1eca7fc640 1 -- 172.21.15.69:0/111020076 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1ec005ef00 con 0x7f1ecc0d0e00 2024-04-25T11:00:02.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.218+0000 7f1ed3c69640 1 -- 172.21.15.69:0/111020076 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f1ecc0d4320 con 0x7f1ecc0d0e00 2024-04-25T11:00:02.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.223+0000 7f1eca7fc640 1 -- 172.21.15.69:0/111020076 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f1ec005e800 con 0x7f1ecc0d0e00 2024-04-25T11:00:02.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.225+0000 7f1ed3c69640 1 -- 172.21.15.69:0/111020076 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1eb80744b0 msgr2=0x7f1eb80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:02.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.225+0000 7f1ed3c69640 1 --2- 172.21.15.69:0/111020076 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1eb80744b0 0x7f1eb80769c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc034000 tx=0x7f1ebc02f000 comp rx=0 tx=0).stop 2024-04-25T11:00:02.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.225+0000 7f1ed3c69640 1 -- 172.21.15.69:0/111020076 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ecc0d0e00 msgr2=0x7f1ecc0cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:02.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.225+0000 7f1ed3c69640 1 --2- 172.21.15.69:0/111020076 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ecc0d0e00 0x7f1ecc0cefa0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1ec000c410 tx=0x7f1ec000c8d0 comp rx=0 tx=0).stop 2024-04-25T11:00:02.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.225+0000 7f1ed3c69640 1 -- 172.21.15.69:0/111020076 shutdown_connections 2024-04-25T11:00:02.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.225+0000 7f1ed3c69640 1 --2- 172.21.15.69:0/111020076 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1eb80744b0 0x7f1eb80769c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:02.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.225+0000 7f1ed3c69640 1 --2- 172.21.15.69:0/111020076 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ecc0d4a50 0x7f1ecc0cf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:02.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.225+0000 7f1ed3c69640 1 --2- 172.21.15.69:0/111020076 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ecc0d0e00 0x7f1ecc0cefa0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:02.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.225+0000 7f1ed3c69640 1 --2- 172.21.15.69:0/111020076 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ecc06c5c0 0x7f1ecc1f97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:02.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.225+0000 7f1ed3c69640 1 -- 172.21.15.69:0/111020076 >> 172.21.15.69:0/111020076 conn(0x7f1ecc0d5d90 msgr2=0x7f1ecc1642c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:02.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.225+0000 7f1ed3c69640 1 -- 172.21.15.69:0/111020076 shutdown_connections 2024-04-25T11:00:02.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.225+0000 7f1ed3c69640 1 -- 172.21.15.69:0/111020076 wait complete. 2024-04-25T11:00:02.236 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-25T11:00:02.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.345+0000 7f0c9e049640 1 Processor -- start 2024-04-25T11:00:02.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.345+0000 7f0c9e049640 1 -- start start 2024-04-25T11:00:02.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.346+0000 7f0c9e049640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0c980b8d30 0x7f0c980d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:02.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.346+0000 7f0c9e049640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0c980cf190 0x7f0c980d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:02.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.346+0000 7f0c9e049640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c980d2a50 con 0x7f0c980b5120 2024-04-25T11:00:02.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.346+0000 7f0c9e049640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f0c980d2d30 con 0x7f0c980b8d30 2024-04-25T11:00:02.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.346+0000 7f0c9e049640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f0c980d3010 con 0x7f0c980cf190 2024-04-25T11:00:02.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.346+0000 7f0c977fe640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0c980b8d30 0x7f0c980d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:02.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.346+0000 7f0c977fe640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0c980b8d30 0x7f0c980d03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48186/0 (socket says 172.21.15.69:48186) 2024-04-25T11:00:02.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.346+0000 7f0c977fe640 1 -- 172.21.15.69:0/2847246300 learned_addr learned my addr 172.21.15.69:0/2847246300 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:02.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.347+0000 7f0c977fe640 1 -- 172.21.15.69:0/2847246300 >> v2:172.21.15.146:3300/0 conn(0x7f0c980cf190 msgr2=0x7f0c980d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:02.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.347+0000 7f0c977fe640 1 --2- 172.21.15.69:0/2847246300 >> v2:172.21.15.146:3300/0 conn(0x7f0c980cf190 0x7f0c980d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:02.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.347+0000 7f0c977fe640 1 -- 172.21.15.69:0/2847246300 >> v1:172.21.15.146:6789/0 conn(0x7f0c980b5120 legacy=0x7f0c9806db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:02.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.347+0000 7f0c977fe640 1 -- 172.21.15.69:0/2847246300 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c980d32f0 con 0x7f0c980b8d30 2024-04-25T11:00:02.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.347+0000 7f0c977fe640 1 --2- 172.21.15.69:0/2847246300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c980b8d30 0x7f0c980d03b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0c980b9b10 tx=0x7f0c8802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6b1d8b831528bf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:02.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.347+0000 7f0c967fc640 1 -- 172.21.15.69:0/2847246300 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c88037070 con 0x7f0c980b8d30 2024-04-25T11:00:02.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.348+0000 7f0c967fc640 1 -- 172.21.15.69:0/2847246300 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c88032440 con 0x7f0c980b8d30 2024-04-25T11:00:02.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.348+0000 7f0c967fc640 1 -- 172.21.15.69:0/2847246300 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c88032810 con 0x7f0c980b8d30 2024-04-25T11:00:02.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.348+0000 7f0c9e049640 1 -- 172.21.15.69:0/2847246300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c980b8d30 msgr2=0x7f0c980d03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:02.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.348+0000 7f0c9e049640 1 --2- 172.21.15.69:0/2847246300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c980b8d30 0x7f0c980d03b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0c980b9b10 tx=0x7f0c8802a230 comp rx=0 tx=0).stop 2024-04-25T11:00:02.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.348+0000 7f0c9e049640 1 -- 172.21.15.69:0/2847246300 shutdown_connections 2024-04-25T11:00:02.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.348+0000 7f0c9e049640 1 --2- 172.21.15.69:0/2847246300 >> v2:172.21.15.146:3300/0 conn(0x7f0c980cf190 0x7f0c980d08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:02.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.348+0000 7f0c9e049640 1 --2- 172.21.15.69:0/2847246300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c980b8d30 0x7f0c980d03b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:02.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.348+0000 7f0c9e049640 1 -- 172.21.15.69:0/2847246300 >> 172.21.15.69:0/2847246300 conn(0x7f0c9808d0a0 msgr2=0x7f0c980d3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:02.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.348+0000 7f0c9e049640 1 -- 172.21.15.69:0/2847246300 shutdown_connections 2024-04-25T11:00:02.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.348+0000 7f0c9e049640 1 -- 172.21.15.69:0/2847246300 wait complete. 2024-04-25T11:00:02.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.350+0000 7f0c9e049640 1 Processor -- start 2024-04-25T11:00:02.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.350+0000 7f0c9e049640 1 -- start start 2024-04-25T11:00:02.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.351+0000 7f0c9e049640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c980b5120 0x7f0c980cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:02.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.351+0000 7f0c9e049640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c980b8d30 0x7f0c981615a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:02.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.352+0000 7f0c9e049640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c980cf190 0x7f0c981f62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:02.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.352+0000 7f0c9e049640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0c980b9280 con 0x7f0c980b5120 2024-04-25T11:00:02.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.352+0000 7f0c9e049640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0c980d4470 con 0x7f0c980b8d30 2024-04-25T11:00:02.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.352+0000 7f0c9e049640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0c980b9400 con 0x7f0c980cf190 2024-04-25T11:00:02.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.352+0000 7f0c977fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c980b5120 0x7f0c980cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:02.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.352+0000 7f0c977fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c980b5120 0x7f0c980cbff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48202/0 (socket says 172.21.15.69:48202) 2024-04-25T11:00:02.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.352+0000 7f0c977fe640 1 -- 172.21.15.69:0/647770671 learned_addr learned my addr 172.21.15.69:0/647770671 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:02.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.352+0000 7f0c97fff640 1 --2- 172.21.15.69:0/647770671 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c980cf190 0x7f0c981f62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:02.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.352+0000 7f0c96ffd640 1 --2- 172.21.15.69:0/647770671 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c980b8d30 0x7f0c981615a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:02.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.352+0000 7f0c977fe640 1 -- 172.21.15.69:0/647770671 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c980cf190 msgr2=0x7f0c981f62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:02.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.352+0000 7f0c977fe640 1 --2- 172.21.15.69:0/647770671 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c980cf190 0x7f0c981f62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:02.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.352+0000 7f0c977fe640 1 -- 172.21.15.69:0/647770671 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c980b8d30 msgr2=0x7f0c981615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:02.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.352+0000 7f0c977fe640 1 --2- 172.21.15.69:0/647770671 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c980b8d30 0x7f0c981615a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:02.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.352+0000 7f0c977fe640 1 -- 172.21.15.69:0/647770671 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c980caf50 con 0x7f0c980b5120 2024-04-25T11:00:02.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.352+0000 7f0c977fe640 1 --2- 172.21.15.69:0/647770671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c980b5120 0x7f0c980cbff0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0c88000c00 tx=0x7f0c880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:02.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.352+0000 7f0c7ffff640 1 -- 172.21.15.69:0/647770671 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c88041070 con 0x7f0c980b5120 2024-04-25T11:00:02.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.352+0000 7f0c7ffff640 1 -- 172.21.15.69:0/647770671 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c8803c870 con 0x7f0c980b5120 2024-04-25T11:00:02.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.352+0000 7f0c9e049640 1 -- 172.21.15.69:0/647770671 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c980cb230 con 0x7f0c980b5120 2024-04-25T11:00:02.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.352+0000 7f0c7ffff640 1 -- 172.21.15.69:0/647770671 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c88037040 con 0x7f0c980b5120 2024-04-25T11:00:02.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.352+0000 7f0c9e049640 1 -- 172.21.15.69:0/647770671 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c980cb7f0 con 0x7f0c980b5120 2024-04-25T11:00:02.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.353+0000 7f0c9e049640 1 -- 172.21.15.69:0/647770671 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c980b87d0 con 0x7f0c980b5120 2024-04-25T11:00:02.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.354+0000 7f0c7ffff640 1 -- 172.21.15.69:0/647770671 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0c88044070 con 0x7f0c980b5120 2024-04-25T11:00:02.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.354+0000 7f0c7ffff640 1 --2- 172.21.15.69:0/647770671 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0c840744b0 0x7f0c840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:02.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.355+0000 7f0c7ffff640 1 -- 172.21.15.69:0/647770671 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7f0c880b59c0 con 0x7f0c980b5120 2024-04-25T11:00:02.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.355+0000 7f0c96ffd640 1 --2- 172.21.15.69:0/647770671 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0c840744b0 0x7f0c840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:02.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.355+0000 7f0c96ffd640 1 --2- 172.21.15.69:0/647770671 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0c840744b0 0x7f0c840769c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0c981fa000 tx=0x7f0c8c009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:02.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.359+0000 7f0c7ffff640 1 -- 172.21.15.69:0/647770671 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0c980b87d0 con 0x7f0c980b5120 2024-04-25T11:00:02.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.601+0000 7f0c9e049640 1 -- 172.21.15.69:0/647770671 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f0c98050740 con 0x7f0c980b5120 2024-04-25T11:00:02.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.606+0000 7f0c7ffff640 1 -- 172.21.15.69:0/647770671 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f0c880b7e20 con 0x7f0c980b5120 2024-04-25T11:00:02.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.608+0000 7f0c9e049640 1 -- 172.21.15.69:0/647770671 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0c840744b0 msgr2=0x7f0c840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:02.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.608+0000 7f0c9e049640 1 --2- 172.21.15.69:0/647770671 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0c840744b0 0x7f0c840769c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0c981fa000 tx=0x7f0c8c009060 comp rx=0 tx=0).stop 2024-04-25T11:00:02.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.608+0000 7f0c9e049640 1 -- 172.21.15.69:0/647770671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c980b5120 msgr2=0x7f0c980cbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:02.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.608+0000 7f0c9e049640 1 --2- 172.21.15.69:0/647770671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c980b5120 0x7f0c980cbff0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0c88000c00 tx=0x7f0c880046c0 comp rx=0 tx=0).stop 2024-04-25T11:00:02.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.608+0000 7f0c9e049640 1 -- 172.21.15.69:0/647770671 shutdown_connections 2024-04-25T11:00:02.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.608+0000 7f0c9e049640 1 --2- 172.21.15.69:0/647770671 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0c840744b0 0x7f0c840769c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:02.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.608+0000 7f0c9e049640 1 --2- 172.21.15.69:0/647770671 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c980cf190 0x7f0c981f62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:02.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.608+0000 7f0c9e049640 1 --2- 172.21.15.69:0/647770671 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c980b8d30 0x7f0c981615a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:02.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.608+0000 7f0c9e049640 1 --2- 172.21.15.69:0/647770671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c980b5120 0x7f0c980cbff0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:02.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.608+0000 7f0c9e049640 1 -- 172.21.15.69:0/647770671 >> 172.21.15.69:0/647770671 conn(0x7f0c9808d0a0 msgr2=0x7f0c980cf5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:02.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.608+0000 7f0c9e049640 1 -- 172.21.15.69:0/647770671 shutdown_connections 2024-04-25T11:00:02.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.608+0000 7f0c9e049640 1 -- 172.21.15.69:0/647770671 wait complete. 2024-04-25T11:00:02.619 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-25T11:00:02.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.726+0000 7fde5c94d640 1 Processor -- start 2024-04-25T11:00:02.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.726+0000 7fde5c94d640 1 -- start start 2024-04-25T11:00:02.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.728+0000 7fde5c94d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fde58144500 0x7fde58070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:02.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.728+0000 7fde5c94d640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde5806f5f0 con 0x7fde58162310 2024-04-25T11:00:02.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.728+0000 7fde5c94d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde5806f8d0 con 0x7fde580cdb90 2024-04-25T11:00:02.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.728+0000 7fde5c94d640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fde58070200 con 0x7fde58144500 2024-04-25T11:00:02.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.728+0000 7fde56d9d640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fde58162310 0x7fde580ce8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:34820/0 (socket says 172.21.15.69:34820) 2024-04-25T11:00:02.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.728+0000 7fde56d9d640 1 -- 172.21.15.69:0/4159972376 learned_addr learned my addr 172.21.15.69:0/4159972376 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:02.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.728+0000 7fde5659c640 1 --2- 172.21.15.69:0/4159972376 >> v2:172.21.15.146:3300/0 conn(0x7fde58144500 0x7fde58070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:02.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.728+0000 7fde55d9b640 1 -- 172.21.15.69:0/4159972376 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 399580788 0 0) 0x7fde5806f5f0 con 0x7fde58162310 2024-04-25T11:00:02.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.728+0000 7fde55d9b640 1 -- 172.21.15.69:0/4159972376 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde44003520 con 0x7fde58162310 2024-04-25T11:00:02.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.728+0000 7fde55d9b640 1 -- 172.21.15.69:0/4159972376 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1814449688 0 0) 0x7fde44003520 con 0x7fde58162310 2024-04-25T11:00:02.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.728+0000 7fde55d9b640 1 -- 172.21.15.69:0/4159972376 >> v2:172.21.15.146:3300/0 conn(0x7fde58144500 msgr2=0x7fde58070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:02.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.728+0000 7fde55d9b640 1 --2- 172.21.15.69:0/4159972376 >> v2:172.21.15.146:3300/0 conn(0x7fde58144500 0x7fde58070690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:02.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.728+0000 7fde55d9b640 1 -- 172.21.15.69:0/4159972376 >> v1:172.21.15.146:6789/0 conn(0x7fde580cdb90 legacy=0x7fde58161c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:02.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.728+0000 7fde55d9b640 1 -- 172.21.15.69:0/4159972376 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde5806fbb0 con 0x7fde58162310 2024-04-25T11:00:02.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.728+0000 7fde55d9b640 1 -- 172.21.15.69:0/4159972376 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7fde500019f0 con 0x7fde58162310 2024-04-25T11:00:02.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.728+0000 7fde55d9b640 1 -- 172.21.15.69:0/4159972376 >> v1:172.21.15.69:6789/0 conn(0x7fde58162310 legacy=0x7fde580ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:02.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.728+0000 7fde55d9b640 1 --2- 172.21.15.69:0/4159972376 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde440039b0 0x7fde44003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:02.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.728+0000 7fde55d9b640 1 --2- 172.21.15.69:0/4159972376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde44004800 0x7fde44004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:02.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.728+0000 7fde55d9b640 1 --2- 172.21.15.69:0/4159972376 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde44005250 0x7fde4400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:02.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.728+0000 7fde55d9b640 1 -- 172.21.15.69:0/4159972376 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fde58070200 con 0x7fde44005250 2024-04-25T11:00:02.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.728+0000 7fde55d9b640 1 -- 172.21.15.69:0/4159972376 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fde4400c1e0 con 0x7fde44004800 2024-04-25T11:00:02.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.728+0000 7fde55d9b640 1 -- 172.21.15.69:0/4159972376 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fde4400c3e0 con 0x7fde440039b0 2024-04-25T11:00:02.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.729+0000 7fde56d9d640 1 --2- 172.21.15.69:0/4159972376 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde44005250 0x7fde4400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:02.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.729+0000 7fde5659c640 1 --2- 172.21.15.69:0/4159972376 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde440039b0 0x7fde44003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:02.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.729+0000 7fde4ffff640 1 --2- 172.21.15.69:0/4159972376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde44004800 0x7fde44004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:02.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.729+0000 7fde56d9d640 1 -- 172.21.15.69:0/4159972376 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde440039b0 msgr2=0x7fde44003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:02.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.729+0000 7fde56d9d640 1 --2- 172.21.15.69:0/4159972376 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde440039b0 0x7fde44003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:02.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.729+0000 7fde56d9d640 1 -- 172.21.15.69:0/4159972376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde44004800 msgr2=0x7fde44004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:02.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.729+0000 7fde56d9d640 1 --2- 172.21.15.69:0/4159972376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde44004800 0x7fde44004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:02.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.729+0000 7fde56d9d640 1 -- 172.21.15.69:0/4159972376 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde44003520 con 0x7fde44005250 2024-04-25T11:00:02.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.729+0000 7fde56d9d640 1 -- 172.21.15.69:0/4159972376 reap_dead start 2024-04-25T11:00:02.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.730+0000 7fde56d9d640 1 --2- 172.21.15.69:0/4159972376 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde44005250 0x7fde4400b9c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fde580701b0 tx=0x7fde44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc42ce24dacff81e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:02.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.730+0000 7fde55d9b640 1 -- 172.21.15.69:0/4159972376 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5000a480 con 0x7fde44005250 2024-04-25T11:00:02.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.730+0000 7fde55d9b640 1 -- 172.21.15.69:0/4159972376 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde5002f070 con 0x7fde44005250 2024-04-25T11:00:02.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.730+0000 7fde55d9b640 1 -- 172.21.15.69:0/4159972376 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5002bc50 con 0x7fde44005250 2024-04-25T11:00:02.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.730+0000 7fde5c94d640 1 -- 172.21.15.69:0/4159972376 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde44005250 msgr2=0x7fde4400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:02.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.730+0000 7fde5c94d640 1 --2- 172.21.15.69:0/4159972376 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde44005250 0x7fde4400b9c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fde580701b0 tx=0x7fde44005830 comp rx=0 tx=0).stop 2024-04-25T11:00:02.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.730+0000 7fde5c94d640 1 -- 172.21.15.69:0/4159972376 shutdown_connections 2024-04-25T11:00:02.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.730+0000 7fde5c94d640 1 --2- 172.21.15.69:0/4159972376 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde44005250 0x7fde4400b9c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:02.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.730+0000 7fde5c94d640 1 -- 172.21.15.69:0/4159972376 >> 172.21.15.69:0/4159972376 conn(0x7fde58073050 msgr2=0x7fde58073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:02.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.730+0000 7fde5c94d640 1 -- 172.21.15.69:0/4159972376 shutdown_connections 2024-04-25T11:00:02.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.730+0000 7fde5c94d640 1 -- 172.21.15.69:0/4159972376 wait complete. 2024-04-25T11:00:02.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.732+0000 7fde5c94d640 1 Processor -- start 2024-04-25T11:00:02.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.732+0000 7fde5c94d640 1 -- start start 2024-04-25T11:00:02.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.734+0000 7fde5c94d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde44005250 0x7fde580cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:02.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.734+0000 7fde5c94d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde581547e0 0x7fde581f6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:02.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.734+0000 7fde5c94d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde580b6440 0x7fde581378c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:02.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.734+0000 7fde5c94d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fde5806f2e0 con 0x7fde44005250 2024-04-25T11:00:02.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.734+0000 7fde5c94d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fde5806f160 con 0x7fde581547e0 2024-04-25T11:00:02.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.734+0000 7fde5c94d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fde580cdfa0 con 0x7fde580b6440 2024-04-25T11:00:02.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.734+0000 7fde5659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde44005250 0x7fde580cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:02.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.734+0000 7fde5659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde44005250 0x7fde580cc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48222/0 (socket says 172.21.15.69:48222) 2024-04-25T11:00:02.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.734+0000 7fde5659c640 1 -- 172.21.15.69:0/2578442080 learned_addr learned my addr 172.21.15.69:0/2578442080 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:02.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.734+0000 7fde4ffff640 1 --2- 172.21.15.69:0/2578442080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde581547e0 0x7fde581f6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:02.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.734+0000 7fde56d9d640 1 --2- 172.21.15.69:0/2578442080 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde580b6440 0x7fde581378c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:02.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.734+0000 7fde5659c640 1 -- 172.21.15.69:0/2578442080 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde580b6440 msgr2=0x7fde581378c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:02.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.734+0000 7fde5659c640 1 --2- 172.21.15.69:0/2578442080 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde580b6440 0x7fde581378c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:02.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.734+0000 7fde5659c640 1 -- 172.21.15.69:0/2578442080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde581547e0 msgr2=0x7fde581f6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:02.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.734+0000 7fde5659c640 1 --2- 172.21.15.69:0/2578442080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde581547e0 0x7fde581f6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:02.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.734+0000 7fde5659c640 1 -- 172.21.15.69:0/2578442080 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde580cbe30 con 0x7fde44005250 2024-04-25T11:00:02.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.734+0000 7fde5659c640 1 --2- 172.21.15.69:0/2578442080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde44005250 0x7fde580cc630 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fde480118e0 tx=0x7fde48011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:02.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.734+0000 7fde4effd640 1 -- 172.21.15.69:0/2578442080 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4800fa90 con 0x7fde44005250 2024-04-25T11:00:02.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.734+0000 7fde4effd640 1 -- 172.21.15.69:0/2578442080 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde4800fc30 con 0x7fde44005250 2024-04-25T11:00:02.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.734+0000 7fde5c94d640 1 -- 172.21.15.69:0/2578442080 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde580b5960 con 0x7fde44005250 2024-04-25T11:00:02.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.734+0000 7fde4effd640 1 -- 172.21.15.69:0/2578442080 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde48014440 con 0x7fde44005250 2024-04-25T11:00:02.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.734+0000 7fde5c94d640 1 -- 172.21.15.69:0/2578442080 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde58155010 con 0x7fde44005250 2024-04-25T11:00:02.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.735+0000 7fde5c94d640 1 -- 172.21.15.69:0/2578442080 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde580b5750 con 0x7fde44005250 2024-04-25T11:00:02.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.736+0000 7fde4effd640 1 -- 172.21.15.69:0/2578442080 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fde48012040 con 0x7fde44005250 2024-04-25T11:00:02.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.736+0000 7fde4effd640 1 --2- 172.21.15.69:0/2578442080 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fde380744e0 0x7fde380769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:02.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.736+0000 7fde4ffff640 1 --2- 172.21.15.69:0/2578442080 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fde380744e0 0x7fde380769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:02.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.736+0000 7fde4effd640 1 -- 172.21.15.69:0/2578442080 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7fde48095450 con 0x7fde44005250 2024-04-25T11:00:02.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.737+0000 7fde4ffff640 1 --2- 172.21.15.69:0/2578442080 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fde380744e0 0x7fde380769f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fde5815e8d0 tx=0x7fde40009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:02.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.741+0000 7fde4effd640 1 -- 172.21.15.69:0/2578442080 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fde48019030 con 0x7fde44005250 2024-04-25T11:00:02.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.981+0000 7fde5c94d640 1 -- 172.21.15.69:0/2578442080 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fde58050740 con 0x7fde44005250 2024-04-25T11:00:02.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.984+0000 7fde4effd640 1 -- 172.21.15.69:0/2578442080 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7fde4805f8e0 con 0x7fde44005250 2024-04-25T11:00:02.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.986+0000 7fde5c94d640 1 -- 172.21.15.69:0/2578442080 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fde380744e0 msgr2=0x7fde380769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:02.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.986+0000 7fde5c94d640 1 --2- 172.21.15.69:0/2578442080 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fde380744e0 0x7fde380769f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fde5815e8d0 tx=0x7fde40009040 comp rx=0 tx=0).stop 2024-04-25T11:00:02.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.986+0000 7fde5c94d640 1 -- 172.21.15.69:0/2578442080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde44005250 msgr2=0x7fde580cc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:02.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.986+0000 7fde5c94d640 1 --2- 172.21.15.69:0/2578442080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde44005250 0x7fde580cc630 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fde480118e0 tx=0x7fde48011da0 comp rx=0 tx=0).stop 2024-04-25T11:00:02.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.986+0000 7fde5c94d640 1 -- 172.21.15.69:0/2578442080 shutdown_connections 2024-04-25T11:00:02.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.986+0000 7fde5c94d640 1 --2- 172.21.15.69:0/2578442080 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fde380744e0 0x7fde380769f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:02.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.986+0000 7fde5c94d640 1 --2- 172.21.15.69:0/2578442080 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde580b6440 0x7fde581378c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:02.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.986+0000 7fde5c94d640 1 --2- 172.21.15.69:0/2578442080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde581547e0 0x7fde581f6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:02.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.986+0000 7fde5c94d640 1 --2- 172.21.15.69:0/2578442080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde44005250 0x7fde580cc630 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:02.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.986+0000 7fde5c94d640 1 -- 172.21.15.69:0/2578442080 >> 172.21.15.69:0/2578442080 conn(0x7fde58073050 msgr2=0x7fde5815c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:02.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.986+0000 7fde5c94d640 1 -- 172.21.15.69:0/2578442080 shutdown_connections 2024-04-25T11:00:02.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:02.986+0000 7fde5c94d640 1 -- 172.21.15.69:0/2578442080 wait complete. 2024-04-25T11:00:02.998 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-25T11:00:03.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.106+0000 7f1e17900640 1 Processor -- start 2024-04-25T11:00:03.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.106+0000 7f1e17900640 1 -- start start 2024-04-25T11:00:03.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.108+0000 7f1e17900640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1e1006d8e0 0x7f1e100d3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:03.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.108+0000 7f1e17900640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f1e1006dcf0 0x7f1e100d3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:03.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.108+0000 7f1e17900640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e100b3c10 con 0x7f1e100d04a0 2024-04-25T11:00:03.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.108+0000 7f1e17900640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f1e100b9520 con 0x7f1e1006dcf0 2024-04-25T11:00:03.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.108+0000 7f1e17900640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f1e100b4820 con 0x7f1e1006d8e0 2024-04-25T11:00:03.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.108+0000 7f1e15e9d640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f1e100d04a0 0x7f1e100d08b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:34834/0 (socket says 172.21.15.69:34834) 2024-04-25T11:00:03.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.108+0000 7f1e15e9d640 1 -- 172.21.15.69:0/4002164834 learned_addr learned my addr 172.21.15.69:0/4002164834 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:03.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.108+0000 7f1e14e9b640 1 --2- 172.21.15.69:0/4002164834 >> v2:172.21.15.69:3300/0 conn(0x7f1e1006dcf0 0x7f1e100d3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:03.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.108+0000 7f1e07fff640 1 -- 172.21.15.69:0/4002164834 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1332649887 0 0) 0x7f1e100b3c10 con 0x7f1e100d04a0 2024-04-25T11:00:03.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.108+0000 7f1e07fff640 1 -- 172.21.15.69:0/4002164834 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1dfc003520 con 0x7f1e100d04a0 2024-04-25T11:00:03.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.108+0000 7f1e1569c640 1 --2- 172.21.15.69:0/4002164834 >> v2:172.21.15.146:3300/0 conn(0x7f1e1006d8e0 0x7f1e100d3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:03.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.108+0000 7f1e14e9b640 1 -- 172.21.15.69:0/4002164834 >> v2:172.21.15.146:3300/0 conn(0x7f1e1006d8e0 msgr2=0x7f1e100d3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:03.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.108+0000 7f1e14e9b640 1 --2- 172.21.15.69:0/4002164834 >> v2:172.21.15.146:3300/0 conn(0x7f1e1006d8e0 0x7f1e100d3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:03.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.108+0000 7f1e14e9b640 1 -- 172.21.15.69:0/4002164834 >> v1:172.21.15.69:6789/0 conn(0x7f1e100d04a0 legacy=0x7f1e100d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:03.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.108+0000 7f1e14e9b640 1 -- 172.21.15.69:0/4002164834 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e100b3ef0 con 0x7f1e1006dcf0 2024-04-25T11:00:03.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.108+0000 7f1e1569c640 1 --2- 172.21.15.69:0/4002164834 >> v2:172.21.15.146:3300/0 conn(0x7f1e1006d8e0 0x7f1e100d3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:00:03.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.109+0000 7f1e14e9b640 1 --2- 172.21.15.69:0/4002164834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e1006dcf0 0x7f1e100d3f60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1e100b38b0 tx=0x7f1e0002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76c16dd14f52057 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:03.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.109+0000 7f1e07fff640 1 -- 172.21.15.69:0/4002164834 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e00037070 con 0x7f1e1006dcf0 2024-04-25T11:00:03.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.109+0000 7f1e07fff640 1 -- 172.21.15.69:0/4002164834 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e00032440 con 0x7f1e1006dcf0 2024-04-25T11:00:03.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.109+0000 7f1e07fff640 1 -- 172.21.15.69:0/4002164834 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0002ae60 con 0x7f1e1006dcf0 2024-04-25T11:00:03.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.110+0000 7f1e17900640 1 -- 172.21.15.69:0/4002164834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e1006dcf0 msgr2=0x7f1e100d3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:03.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.110+0000 7f1e17900640 1 --2- 172.21.15.69:0/4002164834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e1006dcf0 0x7f1e100d3f60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1e100b38b0 tx=0x7f1e0002a110 comp rx=0 tx=0).stop 2024-04-25T11:00:03.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.110+0000 7f1e17900640 1 -- 172.21.15.69:0/4002164834 shutdown_connections 2024-04-25T11:00:03.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.110+0000 7f1e17900640 1 --2- 172.21.15.69:0/4002164834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e1006dcf0 0x7f1e100d3f60 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:03.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.110+0000 7f1e17900640 1 --2- 172.21.15.69:0/4002164834 >> v2:172.21.15.146:3300/0 conn(0x7f1e1006d8e0 0x7f1e100d3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:03.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.110+0000 7f1e17900640 1 -- 172.21.15.69:0/4002164834 >> 172.21.15.69:0/4002164834 conn(0x7f1e1008d110 msgr2=0x7f1e1008d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:03.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.110+0000 7f1e17900640 1 -- 172.21.15.69:0/4002164834 shutdown_connections 2024-04-25T11:00:03.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.110+0000 7f1e17900640 1 -- 172.21.15.69:0/4002164834 wait complete. 2024-04-25T11:00:03.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.112+0000 7f1e17900640 1 Processor -- start 2024-04-25T11:00:03.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.112+0000 7f1e17900640 1 -- start start 2024-04-25T11:00:03.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.113+0000 7f1e17900640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1e1006d8e0 0x7f1e101544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:03.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.113+0000 7f1e17900640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1e1006dcf0 0x7f1e100a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:03.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.113+0000 7f1e17900640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e100d04a0 0x7f1e100713b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:03.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.113+0000 7f1e17900640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1e100b38e0 con 0x7f1e100d04a0 2024-04-25T11:00:03.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.113+0000 7f1e17900640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1e100b44b0 con 0x7f1e1006d8e0 2024-04-25T11:00:03.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.113+0000 7f1e17900640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1e100d0fc0 con 0x7f1e1006dcf0 2024-04-25T11:00:03.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.113+0000 7f1e15e9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e100d04a0 0x7f1e100713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:03.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.113+0000 7f1e15e9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e100d04a0 0x7f1e100713b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48246/0 (socket says 172.21.15.69:48246) 2024-04-25T11:00:03.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.113+0000 7f1e15e9d640 1 -- 172.21.15.69:0/3447539552 learned_addr learned my addr 172.21.15.69:0/3447539552 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:03.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.113+0000 7f1e1569c640 1 --2- 172.21.15.69:0/3447539552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1e1006d8e0 0x7f1e101544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:03.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.113+0000 7f1e15e9d640 1 -- 172.21.15.69:0/3447539552 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1e1006dcf0 msgr2=0x7f1e100a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:03.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.113+0000 7f1e15e9d640 1 --2- 172.21.15.69:0/3447539552 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1e1006dcf0 0x7f1e100a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:03.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.113+0000 7f1e15e9d640 1 -- 172.21.15.69:0/3447539552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1e1006d8e0 msgr2=0x7f1e101544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:03.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.113+0000 7f1e15e9d640 1 --2- 172.21.15.69:0/3447539552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1e1006d8e0 0x7f1e101544d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:03.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.113+0000 7f1e15e9d640 1 -- 172.21.15.69:0/3447539552 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e10154a10 con 0x7f1e100d04a0 2024-04-25T11:00:03.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.114+0000 7f1e15e9d640 1 --2- 172.21.15.69:0/3447539552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e100d04a0 0x7f1e100713b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800c390 tx=0x7f1e0800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:03.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.114+0000 7f1e05ffb640 1 -- 172.21.15.69:0/3447539552 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e08017070 con 0x7f1e100d04a0 2024-04-25T11:00:03.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.114+0000 7f1e05ffb640 1 -- 172.21.15.69:0/3447539552 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e08005ce0 con 0x7f1e100d04a0 2024-04-25T11:00:03.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.114+0000 7f1e17900640 1 -- 172.21.15.69:0/3447539552 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e10154ee0 con 0x7f1e100d04a0 2024-04-25T11:00:03.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.114+0000 7f1e05ffb640 1 -- 172.21.15.69:0/3447539552 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e080072d0 con 0x7f1e100d04a0 2024-04-25T11:00:03.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.114+0000 7f1e17900640 1 -- 172.21.15.69:0/3447539552 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e101554a0 con 0x7f1e100d04a0 2024-04-25T11:00:03.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.115+0000 7f1e17900640 1 -- 172.21.15.69:0/3447539552 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e100d0de0 con 0x7f1e100d04a0 2024-04-25T11:00:03.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.116+0000 7f1e05ffb640 1 -- 172.21.15.69:0/3447539552 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1e0801a070 con 0x7f1e100d04a0 2024-04-25T11:00:03.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.116+0000 7f1e05ffb640 1 --2- 172.21.15.69:0/3447539552 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1df40744b0 0x7f1df40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:03.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.116+0000 7f1e05ffb640 1 -- 172.21.15.69:0/3447539552 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7f1e080940f0 con 0x7f1e100d04a0 2024-04-25T11:00:03.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.116+0000 7f1e1569c640 1 --2- 172.21.15.69:0/3447539552 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1df40744b0 0x7f1df40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:03.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.117+0000 7f1e1569c640 1 --2- 172.21.15.69:0/3447539552 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1df40744b0 0x7f1df40769c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1e100744c0 tx=0x7f1e0c008450 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:03.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.121+0000 7f1e05ffb640 1 -- 172.21.15.69:0/3447539552 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1e100d0de0 con 0x7f1e100d04a0 2024-04-25T11:00:03.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.364+0000 7f1e17900640 1 -- 172.21.15.69:0/3447539552 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f1e10050740 con 0x7f1e100d04a0 2024-04-25T11:00:03.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.366+0000 7f1e05ffb640 1 -- 172.21.15.69:0/3447539552 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7f1e0805e610 con 0x7f1e100d04a0 2024-04-25T11:00:03.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.368+0000 7f1e17900640 1 -- 172.21.15.69:0/3447539552 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1df40744b0 msgr2=0x7f1df40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:03.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.368+0000 7f1e17900640 1 --2- 172.21.15.69:0/3447539552 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1df40744b0 0x7f1df40769c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1e100744c0 tx=0x7f1e0c008450 comp rx=0 tx=0).stop 2024-04-25T11:00:03.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.368+0000 7f1e17900640 1 -- 172.21.15.69:0/3447539552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e100d04a0 msgr2=0x7f1e100713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:03.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.368+0000 7f1e17900640 1 --2- 172.21.15.69:0/3447539552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e100d04a0 0x7f1e100713b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800c390 tx=0x7f1e0800c850 comp rx=0 tx=0).stop 2024-04-25T11:00:03.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.368+0000 7f1e17900640 1 -- 172.21.15.69:0/3447539552 shutdown_connections 2024-04-25T11:00:03.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.368+0000 7f1e17900640 1 --2- 172.21.15.69:0/3447539552 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1df40744b0 0x7f1df40769c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:03.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.368+0000 7f1e17900640 1 --2- 172.21.15.69:0/3447539552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e100d04a0 0x7f1e100713b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:03.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.368+0000 7f1e17900640 1 --2- 172.21.15.69:0/3447539552 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1e1006dcf0 0x7f1e100a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:03.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.368+0000 7f1e17900640 1 --2- 172.21.15.69:0/3447539552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1e1006d8e0 0x7f1e101544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:03.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.368+0000 7f1e17900640 1 -- 172.21.15.69:0/3447539552 >> 172.21.15.69:0/3447539552 conn(0x7f1e1008d110 msgr2=0x7f1e10161290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:03.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.368+0000 7f1e17900640 1 -- 172.21.15.69:0/3447539552 shutdown_connections 2024-04-25T11:00:03.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.368+0000 7f1e17900640 1 -- 172.21.15.69:0/3447539552 wait complete. 2024-04-25T11:00:03.380 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-25T11:00:03.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.487+0000 7f16e78d9640 1 Processor -- start 2024-04-25T11:00:03.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.488+0000 7f16e78d9640 1 -- start start 2024-04-25T11:00:03.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.489+0000 7f16e78d9640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f16e00b6a00 0x7f16e015d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:03.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.489+0000 7f16e78d9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16e006d560 0x7f16e00b4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:03.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.489+0000 7f16e78d9640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16e01611e0 con 0x7f16e015a2f0 2024-04-25T11:00:03.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.489+0000 7f16e78d9640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f16e0160d50 con 0x7f16e00b6a00 2024-04-25T11:00:03.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.489+0000 7f16e78d9640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f16e015a700 con 0x7f16e006d560 2024-04-25T11:00:03.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.489+0000 7f16e5675640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f16e00b6a00 0x7f16e015d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:03.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.489+0000 7f16e5e76640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16e006d560 0x7f16e00b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:03.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.489+0000 7f16e5e76640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16e006d560 0x7f16e00b4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.69:38240/0 (socket says 172.21.15.69:38240) 2024-04-25T11:00:03.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.489+0000 7f16e5e76640 1 -- 172.21.15.69:0/650207141 learned_addr learned my addr 172.21.15.69:0/650207141 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:03.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.489+0000 7f16e4e74640 1 --1- 172.21.15.69:0/650207141 >> v1:172.21.15.146:6789/0 conn(0x7f16e015a2f0 0x7f16e006ce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.69:59784/0 (socket says 172.21.15.69:59784) 2024-04-25T11:00:03.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.490+0000 7f16e5675640 1 -- 172.21.15.69:0/650207141 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16e006d560 msgr2=0x7f16e00b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:03.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.490+0000 7f16e5675640 1 --2- 172.21.15.69:0/650207141 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16e006d560 0x7f16e00b4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:03.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.490+0000 7f16e5675640 1 -- 172.21.15.69:0/650207141 >> v1:172.21.15.146:6789/0 conn(0x7f16e015a2f0 legacy=0x7f16e006ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:03.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.490+0000 7f16d7fff640 1 -- 172.21.15.69:0/650207141 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2055713266 0 0) 0x7f16e01611e0 con 0x7f16e015a2f0 2024-04-25T11:00:03.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.490+0000 7f16e5675640 1 -- 172.21.15.69:0/650207141 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e0160420 con 0x7f16e00b6a00 2024-04-25T11:00:03.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.490+0000 7f16e5e76640 1 --2- 172.21.15.69:0/650207141 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16e006d560 0x7f16e00b4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:00:03.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.490+0000 7f16e5675640 1 --2- 172.21.15.69:0/650207141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16e00b6a00 0x7f16e015d6c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f16dc009d90 tx=0x7f16dc02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc3e858fa86ebe08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:03.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.491+0000 7f16d7fff640 1 -- 172.21.15.69:0/650207141 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc007740 con 0x7f16e00b6a00 2024-04-25T11:00:03.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.491+0000 7f16d7fff640 1 -- 172.21.15.69:0/650207141 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16dc0078e0 con 0x7f16e00b6a00 2024-04-25T11:00:03.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.491+0000 7f16d7fff640 1 -- 172.21.15.69:0/650207141 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc007cb0 con 0x7f16e00b6a00 2024-04-25T11:00:03.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.491+0000 7f16e78d9640 1 -- 172.21.15.69:0/650207141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16e00b6a00 msgr2=0x7f16e015d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:03.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.491+0000 7f16e78d9640 1 --2- 172.21.15.69:0/650207141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16e00b6a00 0x7f16e015d6c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f16dc009d90 tx=0x7f16dc02a660 comp rx=0 tx=0).stop 2024-04-25T11:00:03.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.491+0000 7f16e78d9640 1 -- 172.21.15.69:0/650207141 shutdown_connections 2024-04-25T11:00:03.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.491+0000 7f16e78d9640 1 --2- 172.21.15.69:0/650207141 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16e006d560 0x7f16e00b4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:03.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.491+0000 7f16e78d9640 1 --2- 172.21.15.69:0/650207141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16e00b6a00 0x7f16e015d6c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:03.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.491+0000 7f16e78d9640 1 -- 172.21.15.69:0/650207141 >> 172.21.15.69:0/650207141 conn(0x7f16e00d1ae0 msgr2=0x7f16e00d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:03.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.491+0000 7f16e78d9640 1 -- 172.21.15.69:0/650207141 shutdown_connections 2024-04-25T11:00:03.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.492+0000 7f16e78d9640 1 -- 172.21.15.69:0/650207141 wait complete. 2024-04-25T11:00:03.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.493+0000 7f16e78d9640 1 Processor -- start 2024-04-25T11:00:03.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.493+0000 7f16e78d9640 1 -- start start 2024-04-25T11:00:03.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.494+0000 7f16e78d9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16e006d560 0x7f16e00b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:03.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.494+0000 7f16e78d9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16e00b6a00 0x7f16e00b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:03.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.494+0000 7f16e78d9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16e015a2f0 0x7f16e00b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:03.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.494+0000 7f16e78d9640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f16e00ce580 con 0x7f16e015a2f0 2024-04-25T11:00:03.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.494+0000 7f16e78d9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f16e01609e0 con 0x7f16e006d560 2024-04-25T11:00:03.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.494+0000 7f16e78d9640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f16e00ce700 con 0x7f16e00b6a00 2024-04-25T11:00:03.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.495+0000 7f16e5e76640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16e015a2f0 0x7f16e00b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:03.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.495+0000 7f16e5e76640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16e015a2f0 0x7f16e00b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48250/0 (socket says 172.21.15.69:48250) 2024-04-25T11:00:03.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.495+0000 7f16e5e76640 1 -- 172.21.15.69:0/1626945975 learned_addr learned my addr 172.21.15.69:0/1626945975 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:03.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.495+0000 7f16e5675640 1 --2- 172.21.15.69:0/1626945975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16e006d560 0x7f16e00b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:03.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.495+0000 7f16e4e74640 1 --2- 172.21.15.69:0/1626945975 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16e00b6a00 0x7f16e00b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:03.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.495+0000 7f16e5e76640 1 -- 172.21.15.69:0/1626945975 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16e00b6a00 msgr2=0x7f16e00b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:03.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.495+0000 7f16e5e76640 1 --2- 172.21.15.69:0/1626945975 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16e00b6a00 0x7f16e00b1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:03.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.495+0000 7f16e5e76640 1 -- 172.21.15.69:0/1626945975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16e006d560 msgr2=0x7f16e00b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:03.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.495+0000 7f16e5e76640 1 --2- 172.21.15.69:0/1626945975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16e006d560 0x7f16e00b0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:03.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.495+0000 7f16e5e76640 1 -- 172.21.15.69:0/1626945975 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e0157650 con 0x7f16e015a2f0 2024-04-25T11:00:03.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.495+0000 7f16e5e76640 1 --2- 172.21.15.69:0/1626945975 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16e015a2f0 0x7f16e00b1ff0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f16d800dd40 tx=0x7f16d80094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:03.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.495+0000 7f16d5ffb640 1 -- 172.21.15.69:0/1626945975 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d8009e10 con 0x7f16e015a2f0 2024-04-25T11:00:03.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.495+0000 7f16d5ffb640 1 -- 172.21.15.69:0/1626945975 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16d8004020 con 0x7f16e015a2f0 2024-04-25T11:00:03.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.495+0000 7f16e78d9640 1 -- 172.21.15.69:0/1626945975 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e015f0f0 con 0x7f16e015a2f0 2024-04-25T11:00:03.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.495+0000 7f16d5ffb640 1 -- 172.21.15.69:0/1626945975 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d80043d0 con 0x7f16e015a2f0 2024-04-25T11:00:03.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.496+0000 7f16e78d9640 1 -- 172.21.15.69:0/1626945975 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e015f6b0 con 0x7f16e015a2f0 2024-04-25T11:00:03.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.497+0000 7f16e78d9640 1 -- 172.21.15.69:0/1626945975 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16e01573c0 con 0x7f16e015a2f0 2024-04-25T11:00:03.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.497+0000 7f16d5ffb640 1 -- 172.21.15.69:0/1626945975 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f16d801e020 con 0x7f16e015a2f0 2024-04-25T11:00:03.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.497+0000 7f16d5ffb640 1 --2- 172.21.15.69:0/1626945975 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f16c40744e0 0x7f16c40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:03.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.498+0000 7f16d5ffb640 1 -- 172.21.15.69:0/1626945975 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7f16d80948f0 con 0x7f16e015a2f0 2024-04-25T11:00:03.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.498+0000 7f16e5675640 1 --2- 172.21.15.69:0/1626945975 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f16c40744e0 0x7f16c40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:03.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.498+0000 7f16e5675640 1 --2- 172.21.15.69:0/1626945975 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f16c40744e0 0x7f16c40769f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f16dc034000 tx=0x7f16dc02f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:03.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.502+0000 7f16d5ffb640 1 -- 172.21.15.69:0/1626945975 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f16d805ee20 con 0x7f16e015a2f0 2024-04-25T11:00:03.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.752+0000 7f16e78d9640 1 -- 172.21.15.69:0/1626945975 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f16e00cf750 con 0x7f16e015a2f0 2024-04-25T11:00:03.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.755+0000 7f16d5ffb640 1 -- 172.21.15.69:0/1626945975 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f16e00cf750 con 0x7f16e015a2f0 2024-04-25T11:00:03.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.757+0000 7f16e78d9640 1 -- 172.21.15.69:0/1626945975 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f16c40744e0 msgr2=0x7f16c40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:03.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.757+0000 7f16e78d9640 1 --2- 172.21.15.69:0/1626945975 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f16c40744e0 0x7f16c40769f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f16dc034000 tx=0x7f16dc02f000 comp rx=0 tx=0).stop 2024-04-25T11:00:03.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.757+0000 7f16e78d9640 1 -- 172.21.15.69:0/1626945975 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16e015a2f0 msgr2=0x7f16e00b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:03.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.757+0000 7f16e78d9640 1 --2- 172.21.15.69:0/1626945975 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16e015a2f0 0x7f16e00b1ff0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f16d800dd40 tx=0x7f16d80094f0 comp rx=0 tx=0).stop 2024-04-25T11:00:03.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.757+0000 7f16e78d9640 1 -- 172.21.15.69:0/1626945975 shutdown_connections 2024-04-25T11:00:03.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.757+0000 7f16e78d9640 1 --2- 172.21.15.69:0/1626945975 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f16c40744e0 0x7f16c40769f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:03.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.757+0000 7f16e78d9640 1 --2- 172.21.15.69:0/1626945975 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16e015a2f0 0x7f16e00b1ff0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:03.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.757+0000 7f16e78d9640 1 --2- 172.21.15.69:0/1626945975 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16e00b6a00 0x7f16e00b1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:03.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.757+0000 7f16e78d9640 1 --2- 172.21.15.69:0/1626945975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16e006d560 0x7f16e00b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:03.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.757+0000 7f16e78d9640 1 -- 172.21.15.69:0/1626945975 >> 172.21.15.69:0/1626945975 conn(0x7f16e00d1ae0 msgr2=0x7f16e0158040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:03.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.757+0000 7f16e78d9640 1 -- 172.21.15.69:0/1626945975 shutdown_connections 2024-04-25T11:00:03.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.757+0000 7f16e78d9640 1 -- 172.21.15.69:0/1626945975 wait complete. 2024-04-25T11:00:03.769 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:00:03.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.875+0000 7f0e17651640 1 Processor -- start 2024-04-25T11:00:03.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.875+0000 7f0e17651640 1 -- start start 2024-04-25T11:00:03.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.877+0000 7f0e17651640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0e100d4570 0x7f0e100b4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:03.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.877+0000 7f0e17651640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0e1006e8f0 0x7f0e100d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:03.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.877+0000 7f0e17651640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e1006d870 con 0x7f0e100d09f0 2024-04-25T11:00:03.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.877+0000 7f0e17651640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f0e1006e460 con 0x7f0e1006e8f0 2024-04-25T11:00:03.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.877+0000 7f0e17651640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f0e1006db10 con 0x7f0e100d4570 2024-04-25T11:00:03.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.877+0000 7f0e14bec640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0e1006e8f0 0x7f0e100d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:03.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.877+0000 7f0e14bec640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f0e1006e8f0 0x7f0e100d04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48252/0 (socket says 172.21.15.69:48252) 2024-04-25T11:00:03.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.877+0000 7f0e14bec640 1 -- 172.21.15.69:0/2984132317 learned_addr learned my addr 172.21.15.69:0/2984132317 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:03.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.877+0000 7f0e15bee640 1 --1- 172.21.15.69:0/2984132317 >> v1:172.21.15.69:6789/0 conn(0x7f0e100d09f0 0x7f0e100b2dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:34836/0 (socket says 172.21.15.69:34836) 2024-04-25T11:00:03.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.877+0000 7f0e153ed640 1 --2- 172.21.15.69:0/2984132317 >> v2:172.21.15.146:3300/0 conn(0x7f0e100d4570 0x7f0e100b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:03.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.877+0000 7f0e14bec640 1 -- 172.21.15.69:0/2984132317 >> v2:172.21.15.146:3300/0 conn(0x7f0e100d4570 msgr2=0x7f0e100b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:03.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.877+0000 7f0e14bec640 1 --2- 172.21.15.69:0/2984132317 >> v2:172.21.15.146:3300/0 conn(0x7f0e100d4570 0x7f0e100b4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:03.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.877+0000 7f0e14bec640 1 -- 172.21.15.69:0/2984132317 >> v1:172.21.15.69:6789/0 conn(0x7f0e100d09f0 legacy=0x7f0e100b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:03.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.877+0000 7f0e14bec640 1 -- 172.21.15.69:0/2984132317 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1006e0f0 con 0x7f0e1006e8f0 2024-04-25T11:00:03.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.877+0000 7f0e07fff640 1 -- 172.21.15.69:0/2984132317 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 543189673 0 0) 0x7f0e1006d870 con 0x7f0e100d09f0 2024-04-25T11:00:03.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.878+0000 7f0e14bec640 1 --2- 172.21.15.69:0/2984132317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e1006e8f0 0x7f0e100d04b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0e1006e5e0 tx=0x7f0e0002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af51e0ca13b5196a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:03.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.878+0000 7f0e07fff640 1 -- 172.21.15.69:0/2984132317 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00037070 con 0x7f0e1006e8f0 2024-04-25T11:00:03.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.878+0000 7f0e07fff640 1 -- 172.21.15.69:0/2984132317 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e00032440 con 0x7f0e1006e8f0 2024-04-25T11:00:03.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.878+0000 7f0e07fff640 1 -- 172.21.15.69:0/2984132317 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00032810 con 0x7f0e1006e8f0 2024-04-25T11:00:03.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.879+0000 7f0e17651640 1 -- 172.21.15.69:0/2984132317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e1006e8f0 msgr2=0x7f0e100d04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:03.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.879+0000 7f0e17651640 1 --2- 172.21.15.69:0/2984132317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e1006e8f0 0x7f0e100d04b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0e1006e5e0 tx=0x7f0e0002a1e0 comp rx=0 tx=0).stop 2024-04-25T11:00:03.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.879+0000 7f0e17651640 1 -- 172.21.15.69:0/2984132317 shutdown_connections 2024-04-25T11:00:03.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.879+0000 7f0e17651640 1 --2- 172.21.15.69:0/2984132317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e1006e8f0 0x7f0e100d04b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:03.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.879+0000 7f0e17651640 1 --2- 172.21.15.69:0/2984132317 >> v2:172.21.15.146:3300/0 conn(0x7f0e100d4570 0x7f0e100b4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:03.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.879+0000 7f0e17651640 1 -- 172.21.15.69:0/2984132317 >> 172.21.15.69:0/2984132317 conn(0x7f0e100d3a50 msgr2=0x7f0e100d3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:03.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.879+0000 7f0e17651640 1 -- 172.21.15.69:0/2984132317 shutdown_connections 2024-04-25T11:00:03.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.879+0000 7f0e17651640 1 -- 172.21.15.69:0/2984132317 wait complete. 2024-04-25T11:00:03.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.881+0000 7f0e17651640 1 Processor -- start 2024-04-25T11:00:03.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.881+0000 7f0e17651640 1 -- start start 2024-04-25T11:00:03.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.882+0000 7f0e17651640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e1006e8f0 0x7f0e1014df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:03.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.882+0000 7f0e17651640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e100d09f0 0x7f0e1015f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:03.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.882+0000 7f0e17651640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e100d4570 0x7f0e1015c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:03.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.882+0000 7f0e17651640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0e100b34d0 con 0x7f0e100d09f0 2024-04-25T11:00:03.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.882+0000 7f0e17651640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1006ed00 con 0x7f0e100d4570 2024-04-25T11:00:03.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.882+0000 7f0e17651640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0e100b3650 con 0x7f0e1006e8f0 2024-04-25T11:00:03.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.882+0000 7f0e14bec640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e100d09f0 0x7f0e1015f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:03.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.882+0000 7f0e14bec640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e100d09f0 0x7f0e1015f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48268/0 (socket says 172.21.15.69:48268) 2024-04-25T11:00:03.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.882+0000 7f0e14bec640 1 -- 172.21.15.69:0/3323740944 learned_addr learned my addr 172.21.15.69:0/3323740944 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:03.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.882+0000 7f0e153ed640 1 --2- 172.21.15.69:0/3323740944 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e1006e8f0 0x7f0e1014df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:03.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.882+0000 7f0e15bee640 1 --2- 172.21.15.69:0/3323740944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e100d4570 0x7f0e1015c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:03.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.882+0000 7f0e14bec640 1 -- 172.21.15.69:0/3323740944 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e1006e8f0 msgr2=0x7f0e1014df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:03.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.882+0000 7f0e14bec640 1 --2- 172.21.15.69:0/3323740944 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e1006e8f0 0x7f0e1014df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:03.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.882+0000 7f0e14bec640 1 -- 172.21.15.69:0/3323740944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e100d4570 msgr2=0x7f0e1015c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:03.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.882+0000 7f0e14bec640 1 --2- 172.21.15.69:0/3323740944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e100d4570 0x7f0e1015c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:03.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.882+0000 7f0e14bec640 1 -- 172.21.15.69:0/3323740944 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1014e4d0 con 0x7f0e100d09f0 2024-04-25T11:00:03.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.882+0000 7f0e15bee640 1 --2- 172.21.15.69:0/3323740944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e100d4570 0x7f0e1015c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:03.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.883+0000 7f0e14bec640 1 --2- 172.21.15.69:0/3323740944 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e100d09f0 0x7f0e1015f8c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0e0002a220 tx=0x7f0e000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:03.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.883+0000 7f0e05ffb640 1 -- 172.21.15.69:0/3323740944 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00041020 con 0x7f0e100d09f0 2024-04-25T11:00:03.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.883+0000 7f0e05ffb640 1 -- 172.21.15.69:0/3323740944 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e000306c0 con 0x7f0e100d09f0 2024-04-25T11:00:03.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.883+0000 7f0e05ffb640 1 -- 172.21.15.69:0/3323740944 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00037040 con 0x7f0e100d09f0 2024-04-25T11:00:03.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.883+0000 7f0e17651640 1 -- 172.21.15.69:0/3323740944 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e1015fe00 con 0x7f0e100d09f0 2024-04-25T11:00:03.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.883+0000 7f0e17651640 1 -- 172.21.15.69:0/3323740944 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e101605f0 con 0x7f0e100d09f0 2024-04-25T11:00:03.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.884+0000 7f0e17651640 1 -- 172.21.15.69:0/3323740944 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e10050740 con 0x7f0e100d09f0 2024-04-25T11:00:03.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.884+0000 7f0e05ffb640 1 -- 172.21.15.69:0/3323740944 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0e00044020 con 0x7f0e100d09f0 2024-04-25T11:00:03.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.885+0000 7f0e05ffb640 1 --2- 172.21.15.69:0/3323740944 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0df40744b0 0x7f0df40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:03.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.885+0000 7f0e05ffb640 1 -- 172.21.15.69:0/3323740944 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7f0e000b5680 con 0x7f0e100d09f0 2024-04-25T11:00:03.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.885+0000 7f0e153ed640 1 --2- 172.21.15.69:0/3323740944 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0df40744b0 0x7f0df40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:03.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.885+0000 7f0e153ed640 1 --2- 172.21.15.69:0/3323740944 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0df40744b0 0x7f0df40769c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c003b80 tx=0x7f0e0c009880 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:03.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:03.890+0000 7f0e05ffb640 1 -- 172.21.15.69:0/3323740944 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0e0004b030 con 0x7f0e100d09f0 2024-04-25T11:00:04.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.064+0000 7f0e17651640 1 -- 172.21.15.69:0/3323740944 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e100d1220 con 0x7f0e100d09f0 2024-04-25T11:00:04.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.065+0000 7f0e05ffb640 1 -- 172.21.15.69:0/3323740944 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+12891 (secure 0 0 0) 0x7f0e00080160 con 0x7f0e100d09f0 2024-04-25T11:00:04.065 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:00:04.066 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":27,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T11:00:01.078985+0000","last_up_change":"2024-04-25T10:59:05.546839+0000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:59:06.422674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6825","nonce":1876101065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6827","nonce":1876101065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6831","nonce":1876101065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6829","nonce":1876101065}]},"public_addr":"172.21.15.69:6825/1876101065","cluster_addr":"172.21.15.69:6827/1876101065","heartbeat_back_addr":"172.21.15.69:6831/1876101065","heartbeat_front_addr":"172.21.15.69:6829/1876101065","state":["exists","up"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6801","nonce":1806438563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6803","nonce":1806438563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6807","nonce":1806438563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6805","nonce":1806438563}]},"public_addr":"172.21.15.69:6801/1806438563","cluster_addr":"172.21.15.69:6803/1806438563","heartbeat_back_addr":"172.21.15.69:6807/1806438563","heartbeat_front_addr":"172.21.15.69:6805/1806438563","state":["exists","up"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6817","nonce":1593617983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6819","nonce":1593617983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6823","nonce":1593617983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6821","nonce":1593617983}]},"public_addr":"172.21.15.69:6817/1593617983","cluster_addr":"172.21.15.69:6819/1593617983","heartbeat_back_addr":"172.21.15.69:6823/1593617983","heartbeat_front_addr":"172.21.15.69:6821/1593617983","state":["exists","up"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6809","nonce":3950661294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6811","nonce":3950661294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6815","nonce":3950661294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6813","nonce":3950661294}]},"public_addr":"172.21.15.69:6809/3950661294","cluster_addr":"172.21.15.69:6811/3950661294","heartbeat_back_addr":"172.21.15.69:6815/3950661294","heartbeat_front_addr":"172.21.15.69:6813/3950661294","state":["exists","up"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6801","nonce":2508929872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6803","nonce":2508929872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6807","nonce":2508929872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6805","nonce":2508929872}]},"public_addr":"172.21.15.146:6801/2508929872","cluster_addr":"172.21.15.146:6803/2508929872","heartbeat_back_addr":"172.21.15.146:6807/2508929872","heartbeat_front_addr":"172.21.15.146:6805/2508929872","state":["exists","up"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6811","nonce":1310911138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6816","nonce":1310911138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6823","nonce":1310911138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6821","nonce":1310911138}]},"public_addr":"172.21.15.146:6811/1310911138","cluster_addr":"172.21.15.146:6816/1310911138","heartbeat_back_addr":"172.21.15.146:6823/1310911138","heartbeat_front_addr":"172.21.15.146:6821/1310911138","state":["exists","up"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6810","nonce":634688021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6813","nonce":634688021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6820","nonce":634688021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6817","nonce":634688021}]},"public_addr":"172.21.15.146:6810/634688021","cluster_addr":"172.21.15.146:6813/634688021","heartbeat_back_addr":"172.21.15.146:6820/634688021","heartbeat_front_addr":"172.21.15.146:6817/634688021","state":["exists","up"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6825","nonce":3814215935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6827","nonce":3814215935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6831","nonce":3814215935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6829","nonce":3814215935}]},"public_addr":"172.21.15.146:6825/3814215935","cluster_addr":"172.21.15.146:6827/3814215935","heartbeat_back_addr":"172.21.15.146:6831/3814215935","heartbeat_front_addr":"172.21.15.146:6829/3814215935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.820200+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.850189+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.764560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.866248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.807897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.869881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.782310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.827131+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:00:04.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.067+0000 7f0e17651640 1 -- 172.21.15.69:0/3323740944 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0df40744b0 msgr2=0x7f0df40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:04.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.067+0000 7f0e17651640 1 --2- 172.21.15.69:0/3323740944 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0df40744b0 0x7f0df40769c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c003b80 tx=0x7f0e0c009880 comp rx=0 tx=0).stop 2024-04-25T11:00:04.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.067+0000 7f0e17651640 1 -- 172.21.15.69:0/3323740944 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e100d09f0 msgr2=0x7f0e1015f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:04.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.067+0000 7f0e17651640 1 --2- 172.21.15.69:0/3323740944 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e100d09f0 0x7f0e1015f8c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0e0002a220 tx=0x7f0e000046c0 comp rx=0 tx=0).stop 2024-04-25T11:00:04.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.067+0000 7f0e17651640 1 -- 172.21.15.69:0/3323740944 shutdown_connections 2024-04-25T11:00:04.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.067+0000 7f0e17651640 1 --2- 172.21.15.69:0/3323740944 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0df40744b0 0x7f0df40769c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:04.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.067+0000 7f0e17651640 1 --2- 172.21.15.69:0/3323740944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e100d4570 0x7f0e1015c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:04.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.067+0000 7f0e17651640 1 --2- 172.21.15.69:0/3323740944 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e100d09f0 0x7f0e1015f8c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:04.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.067+0000 7f0e17651640 1 --2- 172.21.15.69:0/3323740944 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e1006e8f0 0x7f0e1014df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:04.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.067+0000 7f0e17651640 1 -- 172.21.15.69:0/3323740944 >> 172.21.15.69:0/3323740944 conn(0x7f0e100d3a50 msgr2=0x7f0e100712e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:04.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.067+0000 7f0e17651640 1 -- 172.21.15.69:0/3323740944 shutdown_connections 2024-04-25T11:00:04.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.067+0000 7f0e17651640 1 -- 172.21.15.69:0/3323740944 wait complete. 2024-04-25T11:00:04.079 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:59:06.422674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:00:04.079 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:00:04.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.187+0000 7ff1ffe27640 1 Processor -- start 2024-04-25T11:00:04.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.188+0000 7ff1ffe27640 1 -- start start 2024-04-25T11:00:04.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.189+0000 7ff1ffe27640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1f80b3240 0x7ff1f80ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:04.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.189+0000 7ff1ffe27640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff1f80cf260 0x7ff1f80d1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:04.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.189+0000 7ff1ffe27640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1f80b21a0 con 0x7ff1f80d2250 2024-04-25T11:00:04.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.189+0000 7ff1ffe27640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7ff1f80b2480 con 0x7ff1f80cf260 2024-04-25T11:00:04.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.189+0000 7ff1ffe27640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f80b2760 con 0x7ff1f80b3240 2024-04-25T11:00:04.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.189+0000 7ff1fdbc3640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1f80b3240 0x7ff1f80ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:04.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.189+0000 7ff1fd3c2640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff1f80cf260 0x7ff1f80d1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:04.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.189+0000 7ff1fd3c2640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff1f80cf260 0x7ff1f80d1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.69:34198/0 (socket says 172.21.15.69:34198) 2024-04-25T11:00:04.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.189+0000 7ff1fdbc3640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1f80b3240 0x7ff1f80ced20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.69:38264/0 (socket says 172.21.15.69:38264) 2024-04-25T11:00:04.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.189+0000 7ff1fd3c2640 1 -- 172.21.15.69:0/3066438037 learned_addr learned my addr 172.21.15.69:0/3066438037 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:04.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.189+0000 7ff1fe3c4640 1 --1- 172.21.15.69:0/3066438037 >> v1:172.21.15.146:6789/0 conn(0x7ff1f80d2250 0x7ff1f806e5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.69:59796/0 (socket says 172.21.15.69:59796) 2024-04-25T11:00:04.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.190+0000 7ff1fdbc3640 1 -- 172.21.15.69:0/3066438037 >> v2:172.21.15.146:3300/0 conn(0x7ff1f80cf260 msgr2=0x7ff1f80d1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:04.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.190+0000 7ff1fdbc3640 1 --2- 172.21.15.69:0/3066438037 >> v2:172.21.15.146:3300/0 conn(0x7ff1f80cf260 0x7ff1f80d1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:04.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.190+0000 7ff1fdbc3640 1 -- 172.21.15.69:0/3066438037 >> v1:172.21.15.146:6789/0 conn(0x7ff1f80d2250 legacy=0x7ff1f806e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:04.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.190+0000 7ff1fdbc3640 1 -- 172.21.15.69:0/3066438037 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f80b2a40 con 0x7ff1f80b3240 2024-04-25T11:00:04.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.190+0000 7ff1fdbc3640 1 --2- 172.21.15.69:0/3066438037 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1f80b3240 0x7ff1f80ced20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009de0 tx=0x7ff1f402a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ade4b8c64eadbc71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:04.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.191+0000 7ff1fcbc1640 1 -- 172.21.15.69:0/3066438037 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4007740 con 0x7ff1f80b3240 2024-04-25T11:00:04.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.191+0000 7ff1fcbc1640 1 -- 172.21.15.69:0/3066438037 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1f40078e0 con 0x7ff1f80b3240 2024-04-25T11:00:04.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.191+0000 7ff1fcbc1640 1 -- 172.21.15.69:0/3066438037 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4007cb0 con 0x7ff1f80b3240 2024-04-25T11:00:04.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.191+0000 7ff1ffe27640 1 -- 172.21.15.69:0/3066438037 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1f80b3240 msgr2=0x7ff1f80ced20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:04.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.191+0000 7ff1ffe27640 1 --2- 172.21.15.69:0/3066438037 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1f80b3240 0x7ff1f80ced20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009de0 tx=0x7ff1f402a6c0 comp rx=0 tx=0).stop 2024-04-25T11:00:04.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.191+0000 7ff1ffe27640 1 -- 172.21.15.69:0/3066438037 shutdown_connections 2024-04-25T11:00:04.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.191+0000 7ff1ffe27640 1 --2- 172.21.15.69:0/3066438037 >> v2:172.21.15.146:3300/0 conn(0x7ff1f80cf260 0x7ff1f80d1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:04.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.191+0000 7ff1ffe27640 1 --2- 172.21.15.69:0/3066438037 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1f80b3240 0x7ff1f80ced20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:04.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.191+0000 7ff1ffe27640 1 -- 172.21.15.69:0/3066438037 >> 172.21.15.69:0/3066438037 conn(0x7ff1f809b7d0 msgr2=0x7ff1f809bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:04.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.192+0000 7ff1ffe27640 1 -- 172.21.15.69:0/3066438037 shutdown_connections 2024-04-25T11:00:04.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.192+0000 7ff1ffe27640 1 -- 172.21.15.69:0/3066438037 wait complete. 2024-04-25T11:00:04.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.193+0000 7ff1ffe27640 1 Processor -- start 2024-04-25T11:00:04.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.193+0000 7ff1ffe27640 1 -- start start 2024-04-25T11:00:04.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.194+0000 7ff1ffe27640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1f80b3240 0x7ff1f814fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:04.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.194+0000 7ff1ffe27640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1f80cf260 0x7ff1f814dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:04.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.194+0000 7ff1ffe27640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1f80d2250 0x7ff1f809cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:04.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.194+0000 7ff1ffe27640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f806ecf0 con 0x7ff1f80d2250 2024-04-25T11:00:04.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.194+0000 7ff1ffe27640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f80d2660 con 0x7ff1f80b3240 2024-04-25T11:00:04.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.194+0000 7ff1ffe27640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f806ee70 con 0x7ff1f80cf260 2024-04-25T11:00:04.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.195+0000 7ff1fe3c4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1f80d2250 0x7ff1f809cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:04.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.195+0000 7ff1fe3c4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1f80d2250 0x7ff1f809cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48270/0 (socket says 172.21.15.69:48270) 2024-04-25T11:00:04.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.195+0000 7ff1fe3c4640 1 -- 172.21.15.69:0/917575915 learned_addr learned my addr 172.21.15.69:0/917575915 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:04.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.195+0000 7ff1fdbc3640 1 --2- 172.21.15.69:0/917575915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1f80b3240 0x7ff1f814fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:04.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.195+0000 7ff1fd3c2640 1 --2- 172.21.15.69:0/917575915 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1f80cf260 0x7ff1f814dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:04.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.195+0000 7ff1fe3c4640 1 -- 172.21.15.69:0/917575915 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1f80cf260 msgr2=0x7ff1f814dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:04.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.195+0000 7ff1fe3c4640 1 --2- 172.21.15.69:0/917575915 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1f80cf260 0x7ff1f814dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:04.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.195+0000 7ff1fe3c4640 1 -- 172.21.15.69:0/917575915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1f80b3240 msgr2=0x7ff1f814fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:04.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.195+0000 7ff1fe3c4640 1 --2- 172.21.15.69:0/917575915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1f80b3240 0x7ff1f814fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:04.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.195+0000 7ff1fe3c4640 1 -- 172.21.15.69:0/917575915 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f814f090 con 0x7ff1f80d2250 2024-04-25T11:00:04.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.195+0000 7ff1fe3c4640 1 --2- 172.21.15.69:0/917575915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1f80d2250 0x7ff1f809cb20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000c3f0 tx=0x7ff1f000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:04.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.195+0000 7ff1ee7fc640 1 -- 172.21.15.69:0/917575915 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f00074e0 con 0x7ff1f80d2250 2024-04-25T11:00:04.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.195+0000 7ff1ee7fc640 1 -- 172.21.15.69:0/917575915 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1f0007680 con 0x7ff1f80d2250 2024-04-25T11:00:04.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.195+0000 7ff1ffe27640 1 -- 172.21.15.69:0/917575915 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f814f370 con 0x7ff1f80d2250 2024-04-25T11:00:04.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.195+0000 7ff1ee7fc640 1 -- 172.21.15.69:0/917575915 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f00049e0 con 0x7ff1f80d2250 2024-04-25T11:00:04.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.195+0000 7ff1ffe27640 1 -- 172.21.15.69:0/917575915 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f814f930 con 0x7ff1f80d2250 2024-04-25T11:00:04.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.196+0000 7ff1ffe27640 1 -- 172.21.15.69:0/917575915 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1f80cf7e0 con 0x7ff1f80d2250 2024-04-25T11:00:04.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.197+0000 7ff1ee7fc640 1 -- 172.21.15.69:0/917575915 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff1f001a020 con 0x7ff1f80d2250 2024-04-25T11:00:04.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.198+0000 7ff1ee7fc640 1 --2- 172.21.15.69:0/917575915 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff1dc0744e0 0x7ff1dc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:04.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.198+0000 7ff1fdbc3640 1 --2- 172.21.15.69:0/917575915 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff1dc0744e0 0x7ff1dc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:04.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.198+0000 7ff1ee7fc640 1 -- 172.21.15.69:0/917575915 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7ff1f0094590 con 0x7ff1f80d2250 2024-04-25T11:00:04.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.198+0000 7ff1fdbc3640 1 --2- 172.21.15.69:0/917575915 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff1dc0744e0 0x7ff1dc0769f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4034000 tx=0x7ff1f402f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:04.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.202+0000 7ff1ee7fc640 1 -- 172.21.15.69:0/917575915 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff1f005f100 con 0x7ff1f80d2250 2024-04-25T11:00:04.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.380+0000 7ff1ffe27640 1 -- 172.21.15.69:0/917575915 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff1f80b1950 con 0x7ff1f80d2250 2024-04-25T11:00:04.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.380+0000 7ff1ee7fc640 1 -- 172.21.15.69:0/917575915 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7ff1f005ea00 con 0x7ff1f80d2250 2024-04-25T11:00:04.381 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-25T11:00:04.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.382+0000 7ff1ffe27640 1 -- 172.21.15.69:0/917575915 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff1dc0744e0 msgr2=0x7ff1dc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:04.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.382+0000 7ff1ffe27640 1 --2- 172.21.15.69:0/917575915 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff1dc0744e0 0x7ff1dc0769f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4034000 tx=0x7ff1f402f000 comp rx=0 tx=0).stop 2024-04-25T11:00:04.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.382+0000 7ff1ffe27640 1 -- 172.21.15.69:0/917575915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1f80d2250 msgr2=0x7ff1f809cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:04.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.382+0000 7ff1ffe27640 1 --2- 172.21.15.69:0/917575915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1f80d2250 0x7ff1f809cb20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000c3f0 tx=0x7ff1f000c8b0 comp rx=0 tx=0).stop 2024-04-25T11:00:04.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.382+0000 7ff1ffe27640 1 -- 172.21.15.69:0/917575915 shutdown_connections 2024-04-25T11:00:04.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.382+0000 7ff1ffe27640 1 --2- 172.21.15.69:0/917575915 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff1dc0744e0 0x7ff1dc0769f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:04.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.382+0000 7ff1ffe27640 1 --2- 172.21.15.69:0/917575915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1f80d2250 0x7ff1f809cb20 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:04.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.382+0000 7ff1ffe27640 1 --2- 172.21.15.69:0/917575915 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1f80cf260 0x7ff1f814dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:04.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.382+0000 7ff1ffe27640 1 --2- 172.21.15.69:0/917575915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1f80b3240 0x7ff1f814fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:04.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.382+0000 7ff1ffe27640 1 -- 172.21.15.69:0/917575915 >> 172.21.15.69:0/917575915 conn(0x7ff1f809b7d0 msgr2=0x7ff1f80b45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:04.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.382+0000 7ff1ffe27640 1 -- 172.21.15.69:0/917575915 shutdown_connections 2024-04-25T11:00:04.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.382+0000 7ff1ffe27640 1 -- 172.21.15.69:0/917575915 wait complete. 2024-04-25T11:00:04.394 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T11:00:04.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.500+0000 7f433895e640 1 Processor -- start 2024-04-25T11:00:04.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.500+0000 7f433895e640 1 -- start start 2024-04-25T11:00:04.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.502+0000 7f433895e640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f43340d4570 0x7f43340b4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:04.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.502+0000 7f433895e640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f43340b2dc0 0x7f43340b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:04.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.502+0000 7f433895e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f433406d870 con 0x7f433406e8f0 2024-04-25T11:00:04.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.502+0000 7f433895e640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f433406e460 con 0x7f43340d4570 2024-04-25T11:00:04.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.502+0000 7f433895e640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f433406db10 con 0x7f43340b2dc0 2024-04-25T11:00:04.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.502+0000 7f433259c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f43340d4570 0x7f43340b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:04.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.502+0000 7f433259c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f43340d4570 0x7f43340b4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48274/0 (socket says 172.21.15.69:48274) 2024-04-25T11:00:04.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.502+0000 7f433259c640 1 -- 172.21.15.69:0/3837785940 learned_addr learned my addr 172.21.15.69:0/3837785940 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:04.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.502+0000 7f4332d9d640 1 --2- 172.21.15.69:0/3837785940 >> v2:172.21.15.146:3300/0 conn(0x7f43340b2dc0 0x7f43340b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:04.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.502+0000 7f433259c640 1 -- 172.21.15.69:0/3837785940 >> v2:172.21.15.146:3300/0 conn(0x7f43340b2dc0 msgr2=0x7f43340b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:04.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.502+0000 7f433259c640 1 --2- 172.21.15.69:0/3837785940 >> v2:172.21.15.146:3300/0 conn(0x7f43340b2dc0 0x7f43340b31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:04.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.502+0000 7f433259c640 1 -- 172.21.15.69:0/3837785940 >> v1:172.21.15.146:6789/0 conn(0x7f433406e8f0 legacy=0x7f43340d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:04.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.502+0000 7f433259c640 1 -- 172.21.15.69:0/3837785940 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f433406e0f0 con 0x7f43340d4570 2024-04-25T11:00:04.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.503+0000 7f433259c640 1 --2- 172.21.15.69:0/3837785940 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43340d4570 0x7f43340b4830 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f433406e5e0 tx=0x7f432402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da4a8f5640ea22e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:04.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.503+0000 7f433159a640 1 -- 172.21.15.69:0/3837785940 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324036070 con 0x7f43340d4570 2024-04-25T11:00:04.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.503+0000 7f433159a640 1 -- 172.21.15.69:0/3837785940 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4324031600 con 0x7f43340d4570 2024-04-25T11:00:04.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.503+0000 7f433159a640 1 -- 172.21.15.69:0/3837785940 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43240319d0 con 0x7f43340d4570 2024-04-25T11:00:04.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.504+0000 7f433895e640 1 -- 172.21.15.69:0/3837785940 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43340d4570 msgr2=0x7f43340b4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:04.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.504+0000 7f433895e640 1 --2- 172.21.15.69:0/3837785940 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43340d4570 0x7f43340b4830 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f433406e5e0 tx=0x7f432402a6f0 comp rx=0 tx=0).stop 2024-04-25T11:00:04.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.504+0000 7f433895e640 1 -- 172.21.15.69:0/3837785940 shutdown_connections 2024-04-25T11:00:04.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.504+0000 7f433895e640 1 --2- 172.21.15.69:0/3837785940 >> v2:172.21.15.146:3300/0 conn(0x7f43340b2dc0 0x7f43340b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:04.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.504+0000 7f433895e640 1 --2- 172.21.15.69:0/3837785940 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43340d4570 0x7f43340b4830 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:04.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.504+0000 7f433895e640 1 -- 172.21.15.69:0/3837785940 >> 172.21.15.69:0/3837785940 conn(0x7f43340d3a50 msgr2=0x7f43340d3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:04.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.504+0000 7f433895e640 1 -- 172.21.15.69:0/3837785940 shutdown_connections 2024-04-25T11:00:04.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.504+0000 7f433895e640 1 -- 172.21.15.69:0/3837785940 wait complete. 2024-04-25T11:00:04.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.506+0000 7f433895e640 1 Processor -- start 2024-04-25T11:00:04.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.506+0000 7f433895e640 1 -- start start 2024-04-25T11:00:04.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.507+0000 7f433895e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f433406e8f0 0x7f433409e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:04.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.507+0000 7f433895e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43340b2dc0 0x7f43340d2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:04.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.507+0000 7f433895e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43340d4570 0x7f43340d3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:04.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.507+0000 7f433895e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f433406ed00 con 0x7f43340d4570 2024-04-25T11:00:04.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.507+0000 7f433895e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f43340d0fb0 con 0x7f43340b2dc0 2024-04-25T11:00:04.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.507+0000 7f433895e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f43340b3710 con 0x7f433406e8f0 2024-04-25T11:00:04.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.507+0000 7f4332d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43340d4570 0x7f43340d3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:04.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.508+0000 7f4332d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43340d4570 0x7f43340d3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48282/0 (socket says 172.21.15.69:48282) 2024-04-25T11:00:04.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.508+0000 7f4332d9d640 1 -- 172.21.15.69:0/3049493799 learned_addr learned my addr 172.21.15.69:0/3049493799 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:04.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.508+0000 7f4331d9b640 1 --2- 172.21.15.69:0/3049493799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43340b2dc0 0x7f43340d2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:04.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.508+0000 7f433259c640 1 --2- 172.21.15.69:0/3049493799 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f433406e8f0 0x7f433409e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:04.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.508+0000 7f4332d9d640 1 -- 172.21.15.69:0/3049493799 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f433406e8f0 msgr2=0x7f433409e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:04.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.508+0000 7f4332d9d640 1 --2- 172.21.15.69:0/3049493799 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f433406e8f0 0x7f433409e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:04.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.508+0000 7f4332d9d640 1 -- 172.21.15.69:0/3049493799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43340b2dc0 msgr2=0x7f43340d2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:04.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.508+0000 7f4332d9d640 1 --2- 172.21.15.69:0/3049493799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43340b2dc0 0x7f43340d2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:04.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.508+0000 7f4332d9d640 1 -- 172.21.15.69:0/3049493799 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4334153230 con 0x7f43340d4570 2024-04-25T11:00:04.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.508+0000 7f4332d9d640 1 --2- 172.21.15.69:0/3049493799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43340d4570 0x7f43340d3340 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f432c00e820 tx=0x7f432c00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:04.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.508+0000 7f4322ffd640 1 -- 172.21.15.69:0/3049493799 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c017070 con 0x7f43340d4570 2024-04-25T11:00:04.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.508+0000 7f4322ffd640 1 -- 172.21.15.69:0/3049493799 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f432c00ee70 con 0x7f43340d4570 2024-04-25T11:00:04.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.508+0000 7f433895e640 1 -- 172.21.15.69:0/3049493799 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43340c77d0 con 0x7f43340d4570 2024-04-25T11:00:04.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.508+0000 7f4322ffd640 1 -- 172.21.15.69:0/3049493799 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c00cb60 con 0x7f43340d4570 2024-04-25T11:00:04.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.508+0000 7f433895e640 1 -- 172.21.15.69:0/3049493799 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43341534f0 con 0x7f43340d4570 2024-04-25T11:00:04.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.509+0000 7f433895e640 1 -- 172.21.15.69:0/3049493799 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4334050740 con 0x7f43340d4570 2024-04-25T11:00:04.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.510+0000 7f4322ffd640 1 -- 172.21.15.69:0/3049493799 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f432c01a070 con 0x7f43340d4570 2024-04-25T11:00:04.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.510+0000 7f4322ffd640 1 --2- 172.21.15.69:0/3049493799 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f43140744b0 0x7f43140769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:04.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.511+0000 7f4322ffd640 1 -- 172.21.15.69:0/3049493799 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7f432c094bb0 con 0x7f43340d4570 2024-04-25T11:00:04.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.511+0000 7f433259c640 1 --2- 172.21.15.69:0/3049493799 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f43140744b0 0x7f43140769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:04.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.511+0000 7f433259c640 1 --2- 172.21.15.69:0/3049493799 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f43140744b0 0x7f43140769c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4324033000 tx=0x7f4324009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:04.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.515+0000 7f4322ffd640 1 -- 172.21.15.69:0/3049493799 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f432c05f160 con 0x7f43340d4570 2024-04-25T11:00:04.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.689+0000 7f433895e640 1 -- 172.21.15.69:0/3049493799 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f433409f4f0 con 0x7f43340d4570 2024-04-25T11:00:04.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.689+0000 7f4322ffd640 1 -- 172.21.15.69:0/3049493799 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v27) ==== 84+0+65 (secure 0 0 0) 0x7f432c063460 con 0x7f43340d4570 2024-04-25T11:00:04.690 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:00:04.690 INFO:teuthology.orchestra.run.smithi069.stdout:[ 2024-04-25T11:00:04.690 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:00:04.691 INFO:teuthology.orchestra.run.smithi069.stdout: "poolnum": 1, 2024-04-25T11:00:04.691 INFO:teuthology.orchestra.run.smithi069.stdout: "poolname": ".mgr" 2024-04-25T11:00:04.691 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:00:04.691 INFO:teuthology.orchestra.run.smithi069.stdout:] 2024-04-25T11:00:04.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.691+0000 7f433895e640 1 -- 172.21.15.69:0/3049493799 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f43140744b0 msgr2=0x7f43140769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:04.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.691+0000 7f433895e640 1 --2- 172.21.15.69:0/3049493799 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f43140744b0 0x7f43140769c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4324033000 tx=0x7f4324009000 comp rx=0 tx=0).stop 2024-04-25T11:00:04.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.691+0000 7f433895e640 1 -- 172.21.15.69:0/3049493799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43340d4570 msgr2=0x7f43340d3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:04.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.691+0000 7f433895e640 1 --2- 172.21.15.69:0/3049493799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43340d4570 0x7f43340d3340 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f432c00e820 tx=0x7f432c00ec20 comp rx=0 tx=0).stop 2024-04-25T11:00:04.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.691+0000 7f433895e640 1 -- 172.21.15.69:0/3049493799 shutdown_connections 2024-04-25T11:00:04.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.691+0000 7f433895e640 1 --2- 172.21.15.69:0/3049493799 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f43140744b0 0x7f43140769c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:04.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.691+0000 7f433895e640 1 --2- 172.21.15.69:0/3049493799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43340d4570 0x7f43340d3340 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:04.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.691+0000 7f433895e640 1 --2- 172.21.15.69:0/3049493799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43340b2dc0 0x7f43340d2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:04.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.691+0000 7f433895e640 1 --2- 172.21.15.69:0/3049493799 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f433406e8f0 0x7f433409e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:04.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.691+0000 7f433895e640 1 -- 172.21.15.69:0/3049493799 >> 172.21.15.69:0/3049493799 conn(0x7f43340d3a50 msgr2=0x7f4334164110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:04.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.691+0000 7f433895e640 1 -- 172.21.15.69:0/3049493799 shutdown_connections 2024-04-25T11:00:04.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.691+0000 7f433895e640 1 -- 172.21.15.69:0/3049493799 wait complete. 2024-04-25T11:00:04.703 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T11:00:04.703 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T11:00:04.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.814+0000 7f831f186640 1 Processor -- start 2024-04-25T11:00:04.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.814+0000 7f831f186640 1 -- start start 2024-04-25T11:00:04.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.815+0000 7f831f186640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8318160550 0x7f831806dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:04.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.815+0000 7f831f186640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83180b3680 0x7f831806c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:04.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.815+0000 7f831f186640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f831806ccc0 con 0x7f83180b76d0 2024-04-25T11:00:04.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.815+0000 7f831f186640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f83180d2ca0 con 0x7f8318160550 2024-04-25T11:00:04.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.815+0000 7f831f186640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f831806d8d0 con 0x7f83180b3680 2024-04-25T11:00:04.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.815+0000 7f831d723640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83180b3680 0x7f831806c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:04.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.815+0000 7f8317fff640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8318160550 0x7f831806dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:04.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.815+0000 7f831cf22640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f83180b76d0 0x7f83180b2f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.69:59820/0 (socket says 172.21.15.69:59820) 2024-04-25T11:00:04.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.815+0000 7f831cf22640 1 -- 172.21.15.69:0/515089754 learned_addr learned my addr 172.21.15.69:0/515089754 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:04.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.816+0000 7f8317fff640 1 -- 172.21.15.69:0/515089754 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83180b3680 msgr2=0x7f831806c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:04.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.816+0000 7f8317fff640 1 --2- 172.21.15.69:0/515089754 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83180b3680 0x7f831806c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:04.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.816+0000 7f8317fff640 1 -- 172.21.15.69:0/515089754 >> v1:172.21.15.146:6789/0 conn(0x7f83180b76d0 legacy=0x7f83180b2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:04.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.816+0000 7f83177fe640 1 -- 172.21.15.69:0/515089754 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4216754539 0 0) 0x7f831806ccc0 con 0x7f83180b76d0 2024-04-25T11:00:04.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.816+0000 7f8317fff640 1 -- 172.21.15.69:0/515089754 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831806cfa0 con 0x7f8318160550 2024-04-25T11:00:04.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.816+0000 7f8317fff640 1 --2- 172.21.15.69:0/515089754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8318160550 0x7f831806dd60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f830c0097a0 tx=0x7f830c02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1fecb1ed59a57bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:04.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.817+0000 7f83177fe640 1 -- 172.21.15.69:0/515089754 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c037070 con 0x7f8318160550 2024-04-25T11:00:04.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.817+0000 7f83177fe640 1 -- 172.21.15.69:0/515089754 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f830c02ae50 con 0x7f8318160550 2024-04-25T11:00:04.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.817+0000 7f83177fe640 1 -- 172.21.15.69:0/515089754 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c032660 con 0x7f8318160550 2024-04-25T11:00:04.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.817+0000 7f831f186640 1 -- 172.21.15.69:0/515089754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8318160550 msgr2=0x7f831806dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:04.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.817+0000 7f831f186640 1 --2- 172.21.15.69:0/515089754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8318160550 0x7f831806dd60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f830c0097a0 tx=0x7f830c02a100 comp rx=0 tx=0).stop 2024-04-25T11:00:04.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.817+0000 7f831f186640 1 -- 172.21.15.69:0/515089754 shutdown_connections 2024-04-25T11:00:04.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.817+0000 7f831f186640 1 --2- 172.21.15.69:0/515089754 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83180b3680 0x7f831806c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:04.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.817+0000 7f831f186640 1 --2- 172.21.15.69:0/515089754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8318160550 0x7f831806dd60 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:04.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.817+0000 7f831f186640 1 -- 172.21.15.69:0/515089754 >> 172.21.15.69:0/515089754 conn(0x7f831809b7d0 msgr2=0x7f83180b6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:04.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.817+0000 7f831f186640 1 -- 172.21.15.69:0/515089754 shutdown_connections 2024-04-25T11:00:04.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.817+0000 7f831f186640 1 -- 172.21.15.69:0/515089754 wait complete. 2024-04-25T11:00:04.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.819+0000 7f831f186640 1 Processor -- start 2024-04-25T11:00:04.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.819+0000 7f831f186640 1 -- start start 2024-04-25T11:00:04.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.820+0000 7f831f186640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83180b3680 0x7f83181f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:04.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.820+0000 7f831f186640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83180b76d0 0x7f83181f61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:04.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.821+0000 7f831f186640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8318160550 0x7f8318161fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:04.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.821+0000 7f831f186640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f83180cfd60 con 0x7f83180b3680 2024-04-25T11:00:04.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.821+0000 7f831f186640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f831806d560 con 0x7f8318160550 2024-04-25T11:00:04.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.821+0000 7f831f186640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f83180cfee0 con 0x7f83180b76d0 2024-04-25T11:00:04.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.821+0000 7f831cf22640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83180b3680 0x7f83181f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:04.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.821+0000 7f831cf22640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83180b3680 0x7f83181f5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48298/0 (socket says 172.21.15.69:48298) 2024-04-25T11:00:04.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.821+0000 7f831cf22640 1 -- 172.21.15.69:0/692558997 learned_addr learned my addr 172.21.15.69:0/692558997 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:04.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.821+0000 7f831d723640 1 --2- 172.21.15.69:0/692558997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8318160550 0x7f8318161fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:04.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.821+0000 7f8317fff640 1 --2- 172.21.15.69:0/692558997 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83180b76d0 0x7f83181f61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:04.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.821+0000 7f831cf22640 1 -- 172.21.15.69:0/692558997 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83180b76d0 msgr2=0x7f83181f61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:04.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.821+0000 7f831cf22640 1 --2- 172.21.15.69:0/692558997 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83180b76d0 0x7f83181f61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:04.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.821+0000 7f831cf22640 1 -- 172.21.15.69:0/692558997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8318160550 msgr2=0x7f8318161fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:04.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.821+0000 7f831cf22640 1 --2- 172.21.15.69:0/692558997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8318160550 0x7f8318161fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:04.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.821+0000 7f831cf22640 1 -- 172.21.15.69:0/692558997 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83180cf160 con 0x7f83180b3680 2024-04-25T11:00:04.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.821+0000 7f831cf22640 1 --2- 172.21.15.69:0/692558997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83180b3680 0x7f83181f5cb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f830800c490 tx=0x7f830800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:04.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.821+0000 7f83157fa640 1 -- 172.21.15.69:0/692558997 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8308017070 con 0x7f83180b3680 2024-04-25T11:00:04.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.821+0000 7f83157fa640 1 -- 172.21.15.69:0/692558997 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8308005d50 con 0x7f83180b3680 2024-04-25T11:00:04.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.821+0000 7f83157fa640 1 -- 172.21.15.69:0/692558997 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8308007310 con 0x7f83180b3680 2024-04-25T11:00:04.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.821+0000 7f831f186640 1 -- 172.21.15.69:0/692558997 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83180cf440 con 0x7f83180b3680 2024-04-25T11:00:04.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.821+0000 7f831f186640 1 -- 172.21.15.69:0/692558997 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83181f6730 con 0x7f83180b3680 2024-04-25T11:00:04.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.823+0000 7f831f186640 1 -- 172.21.15.69:0/692558997 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83181f69c0 con 0x7f83180b3680 2024-04-25T11:00:04.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.823+0000 7f83157fa640 1 -- 172.21.15.69:0/692558997 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f830801a070 con 0x7f83180b3680 2024-04-25T11:00:04.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.823+0000 7f83157fa640 1 --2- 172.21.15.69:0/692558997 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f83040744b0 0x7f83040769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:04.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.823+0000 7f83157fa640 1 -- 172.21.15.69:0/692558997 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7f8308094260 con 0x7f83180b3680 2024-04-25T11:00:04.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.823+0000 7f8317fff640 1 --2- 172.21.15.69:0/692558997 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f83040744b0 0x7f83040769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:04.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.824+0000 7f8317fff640 1 --2- 172.21.15.69:0/692558997 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f83040744b0 0x7f83040769c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f830c034000 tx=0x7f830c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:04.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:04.828+0000 7f83157fa640 1 -- 172.21.15.69:0/692558997 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f830805e6f0 con 0x7f83180b3680 2024-04-25T11:00:05.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.003+0000 7f831f186640 1 -- 172.21.15.69:0/692558997 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f8318050740 con 0x7f83180b3680 2024-04-25T11:00:05.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.233+0000 7f83157fa640 1 -- 172.21.15.69:0/692558997 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v28) ==== 138+0+0 (secure 0 0 0) 0x7f83080629f0 con 0x7f83180b3680 2024-04-25T11:00:05.234 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_metadata' created 2024-04-25T11:00:05.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.236+0000 7f831f186640 1 -- 172.21.15.69:0/692558997 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f83040744b0 msgr2=0x7f83040769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:05.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.236+0000 7f831f186640 1 --2- 172.21.15.69:0/692558997 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f83040744b0 0x7f83040769c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f830c034000 tx=0x7f830c02e000 comp rx=0 tx=0).stop 2024-04-25T11:00:05.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.236+0000 7f831f186640 1 -- 172.21.15.69:0/692558997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83180b3680 msgr2=0x7f83181f5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:05.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.236+0000 7f831f186640 1 --2- 172.21.15.69:0/692558997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83180b3680 0x7f83181f5cb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f830800c490 tx=0x7f830800c950 comp rx=0 tx=0).stop 2024-04-25T11:00:05.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.236+0000 7f831f186640 1 -- 172.21.15.69:0/692558997 shutdown_connections 2024-04-25T11:00:05.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.236+0000 7f831f186640 1 --2- 172.21.15.69:0/692558997 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f83040744b0 0x7f83040769c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:05.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.236+0000 7f831f186640 1 --2- 172.21.15.69:0/692558997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8318160550 0x7f8318161fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:05.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.236+0000 7f831f186640 1 --2- 172.21.15.69:0/692558997 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83180b76d0 0x7f83181f61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:05.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.236+0000 7f831f186640 1 --2- 172.21.15.69:0/692558997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83180b3680 0x7f83181f5cb0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:05.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.236+0000 7f831f186640 1 -- 172.21.15.69:0/692558997 >> 172.21.15.69:0/692558997 conn(0x7f831809b7d0 msgr2=0x7f831815c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:05.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.236+0000 7f831f186640 1 -- 172.21.15.69:0/692558997 shutdown_connections 2024-04-25T11:00:05.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.236+0000 7f831f186640 1 -- 172.21.15.69:0/692558997 wait complete. 2024-04-25T11:00:05.251 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T11:00:05.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.364+0000 7f4f0668c640 1 Processor -- start 2024-04-25T11:00:05.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.364+0000 7f4f0668c640 1 -- start start 2024-04-25T11:00:05.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.365+0000 7f4f0668c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4f00164ad0 0x7f4f000b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:05.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.365+0000 7f4f0668c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f4f00163f30 0x7f4f00164340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:05.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.365+0000 7f4f0668c640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f00164880 con 0x7f4f000d1f30 2024-04-25T11:00:05.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.365+0000 7f4f0668c640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f4f00164ee0 con 0x7f4f00163f30 2024-04-25T11:00:05.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.365+0000 7f4f0668c640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f4f000b31e0 con 0x7f4f00164ad0 2024-04-25T11:00:05.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.366+0000 7f4eff7fe640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f4f00163f30 0x7f4f00164340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:05.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.366+0000 7f4eff7fe640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f4f00163f30 0x7f4f00164340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48302/0 (socket says 172.21.15.69:48302) 2024-04-25T11:00:05.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.366+0000 7f4eff7fe640 1 -- 172.21.15.69:0/2932217589 learned_addr learned my addr 172.21.15.69:0/2932217589 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:05.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.366+0000 7f4efffff640 1 --2- 172.21.15.69:0/2932217589 >> v2:172.21.15.146:3300/0 conn(0x7f4f00164ad0 0x7f4f000b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:05.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.366+0000 7f4eff7fe640 1 -- 172.21.15.69:0/2932217589 >> v2:172.21.15.146:3300/0 conn(0x7f4f00164ad0 msgr2=0x7f4f000b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:05.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.366+0000 7f4eff7fe640 1 --2- 172.21.15.69:0/2932217589 >> v2:172.21.15.146:3300/0 conn(0x7f4f00164ad0 0x7f4f000b2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:05.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.366+0000 7f4eff7fe640 1 -- 172.21.15.69:0/2932217589 >> v1:172.21.15.146:6789/0 conn(0x7f4f000d1f30 legacy=0x7f4f000d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:05.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.366+0000 7f4eff7fe640 1 -- 172.21.15.69:0/2932217589 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f000d3170 con 0x7f4f00163f30 2024-04-25T11:00:05.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.367+0000 7f4eff7fe640 1 --2- 172.21.15.69:0/2932217589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f00163f30 0x7f4f00164340 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4f00074530 tx=0x7f4ef002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8388c3f9054549 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:05.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.367+0000 7f4efeffd640 1 -- 172.21.15.69:0/2932217589 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0037070 con 0x7f4f00163f30 2024-04-25T11:00:05.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.367+0000 7f4efeffd640 1 -- 172.21.15.69:0/2932217589 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ef0032440 con 0x7f4f00163f30 2024-04-25T11:00:05.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.367+0000 7f4efeffd640 1 -- 172.21.15.69:0/2932217589 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0032810 con 0x7f4f00163f30 2024-04-25T11:00:05.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.367+0000 7f4f0668c640 1 -- 172.21.15.69:0/2932217589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f00163f30 msgr2=0x7f4f00164340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:05.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.367+0000 7f4f0668c640 1 --2- 172.21.15.69:0/2932217589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f00163f30 0x7f4f00164340 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4f00074530 tx=0x7f4ef002a270 comp rx=0 tx=0).stop 2024-04-25T11:00:05.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.367+0000 7f4f0668c640 1 -- 172.21.15.69:0/2932217589 shutdown_connections 2024-04-25T11:00:05.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.368+0000 7f4f0668c640 1 --2- 172.21.15.69:0/2932217589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f00163f30 0x7f4f00164340 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:05.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.368+0000 7f4f0668c640 1 --2- 172.21.15.69:0/2932217589 >> v2:172.21.15.146:3300/0 conn(0x7f4f00164ad0 0x7f4f000b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:05.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.368+0000 7f4f0668c640 1 -- 172.21.15.69:0/2932217589 >> 172.21.15.69:0/2932217589 conn(0x7f4f000d3aa0 msgr2=0x7f4f000d3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:05.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.368+0000 7f4f0668c640 1 -- 172.21.15.69:0/2932217589 shutdown_connections 2024-04-25T11:00:05.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.368+0000 7f4f0668c640 1 -- 172.21.15.69:0/2932217589 wait complete. 2024-04-25T11:00:05.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.369+0000 7f4f0668c640 1 Processor -- start 2024-04-25T11:00:05.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.369+0000 7f4f0668c640 1 -- start start 2024-04-25T11:00:05.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.371+0000 7f4f0668c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f000d1f30 0x7f4f000ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:05.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.371+0000 7f4f0668c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f00163f30 0x7f4f001627b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:05.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.371+0000 7f4f0668c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f00164ad0 0x7f4f0015f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:05.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.371+0000 7f4f0668c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0006d9b0 con 0x7f4f00163f30 2024-04-25T11:00:05.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.371+0000 7f4f0668c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0006d6d0 con 0x7f4f00164ad0 2024-04-25T11:00:05.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.371+0000 7f4f0668c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4f000b9430 con 0x7f4f000d1f30 2024-04-25T11:00:05.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.371+0000 7f4eff7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f00163f30 0x7f4f001627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:05.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.371+0000 7f4eff7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f00163f30 0x7f4f001627b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48304/0 (socket says 172.21.15.69:48304) 2024-04-25T11:00:05.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.371+0000 7f4eff7fe640 1 -- 172.21.15.69:0/1081932347 learned_addr learned my addr 172.21.15.69:0/1081932347 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:05.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.371+0000 7f4efffff640 1 --2- 172.21.15.69:0/1081932347 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f000d1f30 0x7f4f000ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:05.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.371+0000 7f4f04c29640 1 --2- 172.21.15.69:0/1081932347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f00164ad0 0x7f4f0015f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:05.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.371+0000 7f4eff7fe640 1 -- 172.21.15.69:0/1081932347 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f000d1f30 msgr2=0x7f4f000ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:05.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.371+0000 7f4eff7fe640 1 --2- 172.21.15.69:0/1081932347 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f000d1f30 0x7f4f000ccbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:05.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.371+0000 7f4eff7fe640 1 -- 172.21.15.69:0/1081932347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f00164ad0 msgr2=0x7f4f0015f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:05.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.371+0000 7f4eff7fe640 1 --2- 172.21.15.69:0/1081932347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f00164ad0 0x7f4f0015f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:05.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.371+0000 7f4eff7fe640 1 -- 172.21.15.69:0/1081932347 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f000cb940 con 0x7f4f00163f30 2024-04-25T11:00:05.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.371+0000 7f4eff7fe640 1 --2- 172.21.15.69:0/1081932347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f00163f30 0x7f4f001627b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4ef002a2b0 tx=0x7f4ef00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:05.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.371+0000 7f4efcff9640 1 -- 172.21.15.69:0/1081932347 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0041020 con 0x7f4f00163f30 2024-04-25T11:00:05.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.371+0000 7f4efcff9640 1 -- 172.21.15.69:0/1081932347 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ef00306c0 con 0x7f4f00163f30 2024-04-25T11:00:05.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.371+0000 7f4f0668c640 1 -- 172.21.15.69:0/1081932347 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f000cb0a0 con 0x7f4f00163f30 2024-04-25T11:00:05.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.372+0000 7f4efcff9640 1 -- 172.21.15.69:0/1081932347 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0037040 con 0x7f4f00163f30 2024-04-25T11:00:05.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.372+0000 7f4f0668c640 1 -- 172.21.15.69:0/1081932347 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f000cb660 con 0x7f4f00163f30 2024-04-25T11:00:05.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.373+0000 7f4f0668c640 1 -- 172.21.15.69:0/1081932347 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f000d4770 con 0x7f4f00163f30 2024-04-25T11:00:05.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.373+0000 7f4efcff9640 1 -- 172.21.15.69:0/1081932347 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4ef0044020 con 0x7f4f00163f30 2024-04-25T11:00:05.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.374+0000 7f4efcff9640 1 --2- 172.21.15.69:0/1081932347 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4ee00744e0 0x7f4ee00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:05.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.374+0000 7f4efcff9640 1 -- 172.21.15.69:0/1081932347 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 5227+0+0 (secure 0 0 0) 0x7f4ef00b58e0 con 0x7f4f00163f30 2024-04-25T11:00:05.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.374+0000 7f4efffff640 1 --2- 172.21.15.69:0/1081932347 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4ee00744e0 0x7f4ee00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:05.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.374+0000 7f4efffff640 1 --2- 172.21.15.69:0/1081932347 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4ee00744e0 0x7f4ee00769f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8006fd0 tx=0x7f4ee8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:05.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.378+0000 7f4efcff9640 1 -- 172.21.15.69:0/1081932347 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4ef004b030 con 0x7f4f00163f30 2024-04-25T11:00:05.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:05.554+0000 7f4f0668c640 1 -- 172.21.15.69:0/1081932347 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f4f00072f90 con 0x7f4f00163f30 2024-04-25T11:00:06.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.240+0000 7f4efcff9640 1 -- 172.21.15.69:0/1081932347 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v29) ==== 170+0+0 (secure 0 0 0) 0x7f4ef0080250 con 0x7f4f00163f30 2024-04-25T11:00:06.240 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_data' created 2024-04-25T11:00:06.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.242+0000 7f4f0668c640 1 -- 172.21.15.69:0/1081932347 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4ee00744e0 msgr2=0x7f4ee00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:06.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.242+0000 7f4f0668c640 1 --2- 172.21.15.69:0/1081932347 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4ee00744e0 0x7f4ee00769f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8006fd0 tx=0x7f4ee8008040 comp rx=0 tx=0).stop 2024-04-25T11:00:06.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.242+0000 7f4f0668c640 1 -- 172.21.15.69:0/1081932347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f00163f30 msgr2=0x7f4f001627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:06.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.242+0000 7f4f0668c640 1 --2- 172.21.15.69:0/1081932347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f00163f30 0x7f4f001627b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4ef002a2b0 tx=0x7f4ef00046c0 comp rx=0 tx=0).stop 2024-04-25T11:00:06.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.242+0000 7f4f0668c640 1 -- 172.21.15.69:0/1081932347 shutdown_connections 2024-04-25T11:00:06.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.242+0000 7f4f0668c640 1 --2- 172.21.15.69:0/1081932347 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4ee00744e0 0x7f4ee00769f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:06.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.242+0000 7f4f0668c640 1 --2- 172.21.15.69:0/1081932347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f00164ad0 0x7f4f0015f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:06.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.242+0000 7f4f0668c640 1 --2- 172.21.15.69:0/1081932347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f00163f30 0x7f4f001627b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:06.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.242+0000 7f4f0668c640 1 --2- 172.21.15.69:0/1081932347 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f000d1f30 0x7f4f000ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:06.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.242+0000 7f4f0668c640 1 -- 172.21.15.69:0/1081932347 >> 172.21.15.69:0/1081932347 conn(0x7f4f000d3aa0 msgr2=0x7f4f000b8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:06.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.243+0000 7f4f0668c640 1 -- 172.21.15.69:0/1081932347 shutdown_connections 2024-04-25T11:00:06.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.243+0000 7f4f0668c640 1 -- 172.21.15.69:0/1081932347 wait complete. 2024-04-25T11:00:06.269 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T11:00:06.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.381+0000 7f185cab6640 1 Processor -- start 2024-04-25T11:00:06.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.381+0000 7f185cab6640 1 -- start start 2024-04-25T11:00:06.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.383+0000 7f185cab6640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f18580b0a80 0x7f18580d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:06.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.383+0000 7f185cab6640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f18580d1c20 0x7f18580d21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:06.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.383+0000 7f185cab6640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18580ce7e0 con 0x7f18580b0670 2024-04-25T11:00:06.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.383+0000 7f185cab6640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f18580d2030 con 0x7f18580b0a80 2024-04-25T11:00:06.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.383+0000 7f185cab6640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f18580b0e90 con 0x7f18580d1c20 2024-04-25T11:00:06.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.383+0000 7f1855d9b640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f18580b0a80 0x7f18580d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:06.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.383+0000 7f1855d9b640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f18580b0a80 0x7f18580d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36450/0 (socket says 172.21.15.69:36450) 2024-04-25T11:00:06.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.383+0000 7f1855d9b640 1 -- 172.21.15.69:0/980362550 learned_addr learned my addr 172.21.15.69:0/980362550 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:06.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.383+0000 7f1856d9d640 1 --2- 172.21.15.69:0/980362550 >> v2:172.21.15.146:3300/0 conn(0x7f18580d1c20 0x7f18580d21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:06.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.383+0000 7f1855d9b640 1 -- 172.21.15.69:0/980362550 >> v2:172.21.15.146:3300/0 conn(0x7f18580d1c20 msgr2=0x7f18580d21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:06.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.383+0000 7f1855d9b640 1 --2- 172.21.15.69:0/980362550 >> v2:172.21.15.146:3300/0 conn(0x7f18580d1c20 0x7f18580d21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:06.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.383+0000 7f1855d9b640 1 -- 172.21.15.69:0/980362550 >> v1:172.21.15.146:6789/0 conn(0x7f18580b0670 legacy=0x7f18580cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:06.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.383+0000 7f1855d9b640 1 -- 172.21.15.69:0/980362550 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18580cea60 con 0x7f18580b0a80 2024-04-25T11:00:06.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.384+0000 7f1856d9d640 1 --2- 172.21.15.69:0/980362550 >> v2:172.21.15.146:3300/0 conn(0x7f18580d1c20 0x7f18580d21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:00:06.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.384+0000 7f1855d9b640 1 --2- 172.21.15.69:0/980362550 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18580b0a80 0x7f18580d16e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1858074d60 tx=0x7f184c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3b593b5dd3fa88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:06.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.384+0000 7f185559a640 1 -- 172.21.15.69:0/980362550 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184c037070 con 0x7f18580b0a80 2024-04-25T11:00:06.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.384+0000 7f185559a640 1 -- 172.21.15.69:0/980362550 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f184c032440 con 0x7f18580b0a80 2024-04-25T11:00:06.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.384+0000 7f185559a640 1 -- 172.21.15.69:0/980362550 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184c032810 con 0x7f18580b0a80 2024-04-25T11:00:06.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.385+0000 7f185cab6640 1 -- 172.21.15.69:0/980362550 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18580b0a80 msgr2=0x7f18580d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:06.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.385+0000 7f185cab6640 1 --2- 172.21.15.69:0/980362550 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18580b0a80 0x7f18580d16e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1858074d60 tx=0x7f184c02a230 comp rx=0 tx=0).stop 2024-04-25T11:00:06.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.385+0000 7f185cab6640 1 -- 172.21.15.69:0/980362550 shutdown_connections 2024-04-25T11:00:06.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.385+0000 7f185cab6640 1 --2- 172.21.15.69:0/980362550 >> v2:172.21.15.146:3300/0 conn(0x7f18580d1c20 0x7f18580d21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:06.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.385+0000 7f185cab6640 1 --2- 172.21.15.69:0/980362550 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18580b0a80 0x7f18580d16e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:06.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.385+0000 7f185cab6640 1 -- 172.21.15.69:0/980362550 >> 172.21.15.69:0/980362550 conn(0x7f18580d3a80 msgr2=0x7f18580d3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:06.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.385+0000 7f185cab6640 1 -- 172.21.15.69:0/980362550 shutdown_connections 2024-04-25T11:00:06.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.385+0000 7f185cab6640 1 -- 172.21.15.69:0/980362550 wait complete. 2024-04-25T11:00:06.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.387+0000 7f185cab6640 1 Processor -- start 2024-04-25T11:00:06.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.387+0000 7f185cab6640 1 -- start start 2024-04-25T11:00:06.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.388+0000 7f185cab6640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18580b0670 0x7f18580b2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:06.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.388+0000 7f185cab6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18580b0a80 0x7f18580b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:06.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.388+0000 7f185cab6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18580d1c20 0x7f1858158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:06.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.388+0000 7f185cab6640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f18580ced20 con 0x7f18580b0a80 2024-04-25T11:00:06.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.388+0000 7f185cab6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f185815f650 con 0x7f18580d1c20 2024-04-25T11:00:06.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.388+0000 7f185cab6640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f18580cf000 con 0x7f18580b0670 2024-04-25T11:00:06.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.388+0000 7f1855d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18580b0a80 0x7f18580b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:06.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.389+0000 7f1855d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18580b0a80 0x7f18580b4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36460/0 (socket says 172.21.15.69:36460) 2024-04-25T11:00:06.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.389+0000 7f1855d9b640 1 -- 172.21.15.69:0/1369719165 learned_addr learned my addr 172.21.15.69:0/1369719165 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:06.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.389+0000 7f1856d9d640 1 --2- 172.21.15.69:0/1369719165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18580d1c20 0x7f1858158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:06.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.389+0000 7f185659c640 1 --2- 172.21.15.69:0/1369719165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18580b0670 0x7f18580b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:06.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.389+0000 7f1855d9b640 1 -- 172.21.15.69:0/1369719165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18580b0670 msgr2=0x7f18580b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:06.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.389+0000 7f1855d9b640 1 --2- 172.21.15.69:0/1369719165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18580b0670 0x7f18580b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:06.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.389+0000 7f1855d9b640 1 -- 172.21.15.69:0/1369719165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18580d1c20 msgr2=0x7f1858158bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:06.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.389+0000 7f1855d9b640 1 --2- 172.21.15.69:0/1369719165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18580d1c20 0x7f1858158bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:06.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.389+0000 7f1855d9b640 1 -- 172.21.15.69:0/1369719165 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18580b5610 con 0x7f18580b0a80 2024-04-25T11:00:06.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.389+0000 7f185659c640 1 --2- 172.21.15.69:0/1369719165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18580b0670 0x7f18580b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:06.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.389+0000 7f1855d9b640 1 --2- 172.21.15.69:0/1369719165 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18580b0a80 0x7f18580b4d30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f184c000c00 tx=0x7f184c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:06.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.389+0000 7f1846ffd640 1 -- 172.21.15.69:0/1369719165 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184c037040 con 0x7f18580b0a80 2024-04-25T11:00:06.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.389+0000 7f1846ffd640 1 -- 172.21.15.69:0/1369719165 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f184c042070 con 0x7f18580b0a80 2024-04-25T11:00:06.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.389+0000 7f185cab6640 1 -- 172.21.15.69:0/1369719165 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f185806d8e0 con 0x7f18580b0a80 2024-04-25T11:00:06.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.389+0000 7f1846ffd640 1 -- 172.21.15.69:0/1369719165 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184c030bb0 con 0x7f18580b0a80 2024-04-25T11:00:06.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.389+0000 7f185cab6640 1 -- 172.21.15.69:0/1369719165 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f185806dea0 con 0x7f18580b0a80 2024-04-25T11:00:06.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.390+0000 7f185cab6640 1 -- 172.21.15.69:0/1369719165 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f185806d6d0 con 0x7f18580b0a80 2024-04-25T11:00:06.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.391+0000 7f1846ffd640 1 -- 172.21.15.69:0/1369719165 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f184c047460 con 0x7f18580b0a80 2024-04-25T11:00:06.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.391+0000 7f1846ffd640 1 --2- 172.21.15.69:0/1369719165 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f18380744e0 0x7f18380769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:06.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.392+0000 7f1846ffd640 1 -- 172.21.15.69:0/1369719165 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5609+0+0 (secure 0 0 0) 0x7f184c0b5de0 con 0x7f18580b0a80 2024-04-25T11:00:06.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.392+0000 7f185659c640 1 --2- 172.21.15.69:0/1369719165 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f18380744e0 0x7f18380769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:06.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.392+0000 7f185659c640 1 --2- 172.21.15.69:0/1369719165 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f18380744e0 0x7f18380769f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f18580b6570 tx=0x7f1848009570 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:06.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.397+0000 7f1846ffd640 1 -- 172.21.15.69:0/1369719165 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f18580ce7e0 con 0x7f18580b0a80 2024-04-25T11:00:06.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:06.607+0000 7f185cab6640 1 -- 172.21.15.69:0/1369719165 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f1858050740 con 0x7f18580b0a80 2024-04-25T11:00:07.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.249+0000 7f1846ffd640 1 -- 172.21.15.69:0/1369719165 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v11) ==== 172+0+0 (secure 0 0 0) 0x7f184c0805d0 con 0x7f18580b0a80 2024-04-25T11:00:07.252 INFO:teuthology.orchestra.run.smithi069.stderr:new fs with metadata pool 4 and data pool 5 2024-04-25T11:00:07.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.257+0000 7f185cab6640 1 -- 172.21.15.69:0/1369719165 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f18380744e0 msgr2=0x7f18380769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:07.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.257+0000 7f185cab6640 1 --2- 172.21.15.69:0/1369719165 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f18380744e0 0x7f18380769f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f18580b6570 tx=0x7f1848009570 comp rx=0 tx=0).stop 2024-04-25T11:00:07.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.257+0000 7f185cab6640 1 -- 172.21.15.69:0/1369719165 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18580b0a80 msgr2=0x7f18580b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:07.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.257+0000 7f185cab6640 1 --2- 172.21.15.69:0/1369719165 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18580b0a80 0x7f18580b4d30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f184c000c00 tx=0x7f184c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:00:07.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.261+0000 7f185cab6640 1 -- 172.21.15.69:0/1369719165 shutdown_connections 2024-04-25T11:00:07.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.261+0000 7f185cab6640 1 --2- 172.21.15.69:0/1369719165 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f18380744e0 0x7f18380769f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:07.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.261+0000 7f185cab6640 1 --2- 172.21.15.69:0/1369719165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18580d1c20 0x7f1858158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:07.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.261+0000 7f185cab6640 1 --2- 172.21.15.69:0/1369719165 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18580b0a80 0x7f18580b4d30 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:07.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.261+0000 7f185cab6640 1 --2- 172.21.15.69:0/1369719165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18580b0670 0x7f18580b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:07.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.261+0000 7f185cab6640 1 -- 172.21.15.69:0/1369719165 >> 172.21.15.69:0/1369719165 conn(0x7f18580d3a80 msgr2=0x7f18580b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:07.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.266+0000 7f185cab6640 1 -- 172.21.15.69:0/1369719165 shutdown_connections 2024-04-25T11:00:07.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.266+0000 7f185cab6640 1 -- 172.21.15.69:0/1369719165 wait complete. 2024-04-25T11:00:07.284 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:00:07.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.390+0000 7fb85f7c7640 1 Processor -- start 2024-04-25T11:00:07.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.391+0000 7fb85f7c7640 1 -- start start 2024-04-25T11:00:07.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.392+0000 7fb85f7c7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb858162930 0x7fb8580d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:07.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.392+0000 7fb85f7c7640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8580d2a90 con 0x7fb8580b3680 2024-04-25T11:00:07.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.392+0000 7fb85f7c7640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8580d27b0 con 0x7fb8580d3e20 2024-04-25T11:00:07.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.392+0000 7fb85f7c7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb858161ac0 con 0x7fb858162930 2024-04-25T11:00:07.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.392+0000 7fb85cd62640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fb8580b3680 0x7fb8580d3710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:38834/0 (socket says 172.21.15.69:38834) 2024-04-25T11:00:07.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.392+0000 7fb85cd62640 1 -- 172.21.15.69:0/1595476373 learned_addr learned my addr 172.21.15.69:0/1595476373 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:07.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.392+0000 7fb85d563640 1 --2- 172.21.15.69:0/1595476373 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb858162930 0x7fb8580d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:07.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.392+0000 7fb84ffff640 1 -- 172.21.15.69:0/1595476373 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3659573626 0 0) 0x7fb8580d2a90 con 0x7fb8580b3680 2024-04-25T11:00:07.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.392+0000 7fb84ffff640 1 -- 172.21.15.69:0/1595476373 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb844003520 con 0x7fb8580b3680 2024-04-25T11:00:07.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.393+0000 7fb84ffff640 1 -- 172.21.15.69:0/1595476373 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2412018246 0 0) 0x7fb844003520 con 0x7fb8580b3680 2024-04-25T11:00:07.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.393+0000 7fb84ffff640 1 -- 172.21.15.69:0/1595476373 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb858162930 msgr2=0x7fb8580d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:07.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.393+0000 7fb84ffff640 1 --2- 172.21.15.69:0/1595476373 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb858162930 0x7fb8580d2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:07.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.393+0000 7fb84ffff640 1 -- 172.21.15.69:0/1595476373 >> v1:172.21.15.146:6789/0 conn(0x7fb8580d3e20 legacy=0x7fb8580ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:07.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.393+0000 7fb84ffff640 1 -- 172.21.15.69:0/1595476373 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb858161e50 con 0x7fb8580b3680 2024-04-25T11:00:07.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.393+0000 7fb84ffff640 1 -- 172.21.15.69:0/1595476373 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7fb848001670 con 0x7fb8580b3680 2024-04-25T11:00:07.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.393+0000 7fb84ffff640 1 -- 172.21.15.69:0/1595476373 >> v1:172.21.15.69:6789/0 conn(0x7fb8580b3680 legacy=0x7fb8580d3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:07.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.393+0000 7fb84ffff640 1 --2- 172.21.15.69:0/1595476373 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8440039e0 0x7fb844003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:07.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.393+0000 7fb84ffff640 1 --2- 172.21.15.69:0/1595476373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb844004350 0x7fb844004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:07.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.393+0000 7fb84ffff640 1 --2- 172.21.15.69:0/1595476373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8440050d0 0x7fb84400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:07.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.393+0000 7fb84ffff640 1 -- 172.21.15.69:0/1595476373 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb858161ac0 con 0x7fb8440050d0 2024-04-25T11:00:07.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.393+0000 7fb84ffff640 1 -- 172.21.15.69:0/1595476373 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb84400c030 con 0x7fb844004350 2024-04-25T11:00:07.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.393+0000 7fb84ffff640 1 -- 172.21.15.69:0/1595476373 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb84400c230 con 0x7fb8440039e0 2024-04-25T11:00:07.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.393+0000 7fb85dd64640 1 --2- 172.21.15.69:0/1595476373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8440050d0 0x7fb84400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:07.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.393+0000 7fb85d563640 1 --2- 172.21.15.69:0/1595476373 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8440039e0 0x7fb844003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:07.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.393+0000 7fb85cd62640 1 --2- 172.21.15.69:0/1595476373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb844004350 0x7fb844004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:07.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.393+0000 7fb85dd64640 1 -- 172.21.15.69:0/1595476373 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8440039e0 msgr2=0x7fb844003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:07.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.393+0000 7fb85dd64640 1 --2- 172.21.15.69:0/1595476373 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8440039e0 0x7fb844003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:07.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.393+0000 7fb85dd64640 1 -- 172.21.15.69:0/1595476373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb844004350 msgr2=0x7fb844004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:07.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.393+0000 7fb85dd64640 1 --2- 172.21.15.69:0/1595476373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb844004350 0x7fb844004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:07.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.393+0000 7fb85dd64640 1 -- 172.21.15.69:0/1595476373 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb844003520 con 0x7fb8440050d0 2024-04-25T11:00:07.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.393+0000 7fb85cd62640 1 --2- 172.21.15.69:0/1595476373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb844004350 0x7fb844004880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:07.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.394+0000 7fb85dd64640 1 --2- 172.21.15.69:0/1595476373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8440050d0 0x7fb84400b9c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb8580d00a0 tx=0x7fb844005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a08d75294bda8aba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:07.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.394+0000 7fb84ffff640 1 -- 172.21.15.69:0/1595476373 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb850037070 con 0x7fb8440050d0 2024-04-25T11:00:07.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.394+0000 7fb84ffff640 1 -- 172.21.15.69:0/1595476373 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb850032440 con 0x7fb8440050d0 2024-04-25T11:00:07.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.394+0000 7fb84ffff640 1 -- 172.21.15.69:0/1595476373 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb850001b30 con 0x7fb8440050d0 2024-04-25T11:00:07.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.395+0000 7fb85f7c7640 1 -- 172.21.15.69:0/1595476373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8440050d0 msgr2=0x7fb84400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:07.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.395+0000 7fb85f7c7640 1 --2- 172.21.15.69:0/1595476373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8440050d0 0x7fb84400b9c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb8580d00a0 tx=0x7fb844005830 comp rx=0 tx=0).stop 2024-04-25T11:00:07.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.395+0000 7fb85dd64640 1 -- 172.21.15.69:0/1595476373 reap_dead start 2024-04-25T11:00:07.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.395+0000 7fb85f7c7640 1 -- 172.21.15.69:0/1595476373 shutdown_connections 2024-04-25T11:00:07.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.395+0000 7fb85f7c7640 1 -- 172.21.15.69:0/1595476373 >> 172.21.15.69:0/1595476373 conn(0x7fb85808d110 msgr2=0x7fb85808d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:07.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.395+0000 7fb85f7c7640 1 -- 172.21.15.69:0/1595476373 shutdown_connections 2024-04-25T11:00:07.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.395+0000 7fb85f7c7640 1 -- 172.21.15.69:0/1595476373 wait complete. 2024-04-25T11:00:07.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.397+0000 7fb85f7c7640 1 Processor -- start 2024-04-25T11:00:07.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.397+0000 7fb85f7c7640 1 -- start start 2024-04-25T11:00:07.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.398+0000 7fb85f7c7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8580cdaa0 0x7fb8581f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:07.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.398+0000 7fb85f7c7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb85815fd70 0x7fb858160180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:07.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.398+0000 7fb85f7c7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8580b1610 0x7fb8580b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:07.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.398+0000 7fb85f7c7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb8580b3a90 con 0x7fb85815fd70 2024-04-25T11:00:07.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.398+0000 7fb85f7c7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb858162410 con 0x7fb8580b1610 2024-04-25T11:00:07.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.398+0000 7fb85f7c7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb8580b93f0 con 0x7fb8580cdaa0 2024-04-25T11:00:07.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.398+0000 7fb85cd62640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb85815fd70 0x7fb858160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:07.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.398+0000 7fb85cd62640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb85815fd70 0x7fb858160180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36480/0 (socket says 172.21.15.69:36480) 2024-04-25T11:00:07.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.398+0000 7fb85cd62640 1 -- 172.21.15.69:0/2784756953 learned_addr learned my addr 172.21.15.69:0/2784756953 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:07.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.398+0000 7fb85d563640 1 --2- 172.21.15.69:0/2784756953 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8580cdaa0 0x7fb8581f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:07.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.398+0000 7fb85dd64640 1 --2- 172.21.15.69:0/2784756953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8580b1610 0x7fb8580b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:07.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.398+0000 7fb85cd62640 1 -- 172.21.15.69:0/2784756953 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8580cdaa0 msgr2=0x7fb8581f46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:07.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.398+0000 7fb85cd62640 1 --2- 172.21.15.69:0/2784756953 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8580cdaa0 0x7fb8581f46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:07.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.398+0000 7fb85cd62640 1 -- 172.21.15.69:0/2784756953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8580b1610 msgr2=0x7fb8580b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:07.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.398+0000 7fb85cd62640 1 --2- 172.21.15.69:0/2784756953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8580b1610 0x7fb8580b1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:07.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.398+0000 7fb85cd62640 1 -- 172.21.15.69:0/2784756953 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8581f4c10 con 0x7fb85815fd70 2024-04-25T11:00:07.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.398+0000 7fb85cd62640 1 --2- 172.21.15.69:0/2784756953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb85815fd70 0x7fb858160180 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb84800eb20 tx=0x7fb84800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:07.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.399+0000 7fb84dffb640 1 -- 172.21.15.69:0/2784756953 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84800c870 con 0x7fb85815fd70 2024-04-25T11:00:07.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.399+0000 7fb84dffb640 1 -- 172.21.15.69:0/2784756953 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb848018070 con 0x7fb85815fd70 2024-04-25T11:00:07.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.399+0000 7fb85f7c7640 1 -- 172.21.15.69:0/2784756953 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8581606c0 con 0x7fb85815fd70 2024-04-25T11:00:07.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.399+0000 7fb84dffb640 1 -- 172.21.15.69:0/2784756953 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84800cc10 con 0x7fb85815fd70 2024-04-25T11:00:07.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.399+0000 7fb85f7c7640 1 -- 172.21.15.69:0/2784756953 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb85815e370 con 0x7fb85815fd70 2024-04-25T11:00:07.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.400+0000 7fb85f7c7640 1 -- 172.21.15.69:0/2784756953 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8580d1ed0 con 0x7fb85815fd70 2024-04-25T11:00:07.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.400+0000 7fb84dffb640 1 -- 172.21.15.69:0/2784756953 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb84801d460 con 0x7fb85815fd70 2024-04-25T11:00:07.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.401+0000 7fb84dffb640 1 --2- 172.21.15.69:0/2784756953 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb83c0744b0 0x7fb83c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:07.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.401+0000 7fb84dffb640 1 -- 172.21.15.69:0/2784756953 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7fb848094e40 con 0x7fb85815fd70 2024-04-25T11:00:07.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.401+0000 7fb85d563640 1 --2- 172.21.15.69:0/2784756953 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb83c0744b0 0x7fb83c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:07.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.401+0000 7fb85d563640 1 --2- 172.21.15.69:0/2784756953 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb83c0744b0 0x7fb83c0769c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb85815cc80 tx=0x7fb854009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:07.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.405+0000 7fb84dffb640 1 -- 172.21.15.69:0/2784756953 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb8581606c0 con 0x7fb85815fd70 2024-04-25T11:00:07.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.580+0000 7fb85f7c7640 1 -- 172.21.15.69:0/2784756953 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb85815f090 con 0x7fb85815fd70 2024-04-25T11:00:07.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.581+0000 7fb84dffb640 1 -- 172.21.15.69:0/2784756953 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) ==== 74+0+16682 (secure 0 0 0) 0x7fb8581f4c10 con 0x7fb85815fd70 2024-04-25T11:00:07.582 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:00:07.582 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":30,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T11:00:06.608806+0000","last_up_change":"2024-04-25T10:59:05.546839+0000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:59:06.422674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:00:05.007593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T11:00:05.558880+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6825","nonce":1876101065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6827","nonce":1876101065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6831","nonce":1876101065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6829","nonce":1876101065}]},"public_addr":"172.21.15.69:6825/1876101065","cluster_addr":"172.21.15.69:6827/1876101065","heartbeat_back_addr":"172.21.15.69:6831/1876101065","heartbeat_front_addr":"172.21.15.69:6829/1876101065","state":["exists","up"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6801","nonce":1806438563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6803","nonce":1806438563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6807","nonce":1806438563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6805","nonce":1806438563}]},"public_addr":"172.21.15.69:6801/1806438563","cluster_addr":"172.21.15.69:6803/1806438563","heartbeat_back_addr":"172.21.15.69:6807/1806438563","heartbeat_front_addr":"172.21.15.69:6805/1806438563","state":["exists","up"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6817","nonce":1593617983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6819","nonce":1593617983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6823","nonce":1593617983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6821","nonce":1593617983}]},"public_addr":"172.21.15.69:6817/1593617983","cluster_addr":"172.21.15.69:6819/1593617983","heartbeat_back_addr":"172.21.15.69:6823/1593617983","heartbeat_front_addr":"172.21.15.69:6821/1593617983","state":["exists","up"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6809","nonce":3950661294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6811","nonce":3950661294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6815","nonce":3950661294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6813","nonce":3950661294}]},"public_addr":"172.21.15.69:6809/3950661294","cluster_addr":"172.21.15.69:6811/3950661294","heartbeat_back_addr":"172.21.15.69:6815/3950661294","heartbeat_front_addr":"172.21.15.69:6813/3950661294","state":["exists","up"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6801","nonce":2508929872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6803","nonce":2508929872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6807","nonce":2508929872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6805","nonce":2508929872}]},"public_addr":"172.21.15.146:6801/2508929872","cluster_addr":"172.21.15.146:6803/2508929872","heartbeat_back_addr":"172.21.15.146:6807/2508929872","heartbeat_front_addr":"172.21.15.146:6805/2508929872","state":["exists","up"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6811","nonce":1310911138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6816","nonce":1310911138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6823","nonce":1310911138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6821","nonce":1310911138}]},"public_addr":"172.21.15.146:6811/1310911138","cluster_addr":"172.21.15.146:6816/1310911138","heartbeat_back_addr":"172.21.15.146:6823/1310911138","heartbeat_front_addr":"172.21.15.146:6821/1310911138","state":["exists","up"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6810","nonce":634688021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6813","nonce":634688021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6820","nonce":634688021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6817","nonce":634688021}]},"public_addr":"172.21.15.146:6810/634688021","cluster_addr":"172.21.15.146:6813/634688021","heartbeat_back_addr":"172.21.15.146:6820/634688021","heartbeat_front_addr":"172.21.15.146:6817/634688021","state":["exists","up"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6825","nonce":3814215935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6827","nonce":3814215935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6831","nonce":3814215935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6829","nonce":3814215935}]},"public_addr":"172.21.15.146:6825/3814215935","cluster_addr":"172.21.15.146:6827/3814215935","heartbeat_back_addr":"172.21.15.146:6831/3814215935","heartbeat_front_addr":"172.21.15.146:6829/3814215935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.820200+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.850189+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.764560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.866248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.807897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.869881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.782310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.827131+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:00:07.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.583+0000 7fb85f7c7640 1 -- 172.21.15.69:0/2784756953 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb83c0744b0 msgr2=0x7fb83c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:07.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.583+0000 7fb85f7c7640 1 --2- 172.21.15.69:0/2784756953 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb83c0744b0 0x7fb83c0769c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb85815cc80 tx=0x7fb854009190 comp rx=0 tx=0).stop 2024-04-25T11:00:07.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.583+0000 7fb85f7c7640 1 -- 172.21.15.69:0/2784756953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb85815fd70 msgr2=0x7fb858160180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:07.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.583+0000 7fb85f7c7640 1 --2- 172.21.15.69:0/2784756953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb85815fd70 0x7fb858160180 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb84800eb20 tx=0x7fb84800ef20 comp rx=0 tx=0).stop 2024-04-25T11:00:07.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.583+0000 7fb85f7c7640 1 -- 172.21.15.69:0/2784756953 shutdown_connections 2024-04-25T11:00:07.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.583+0000 7fb85f7c7640 1 --2- 172.21.15.69:0/2784756953 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb83c0744b0 0x7fb83c0769c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:07.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.583+0000 7fb85f7c7640 1 --2- 172.21.15.69:0/2784756953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8580b1610 0x7fb8580b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:07.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.583+0000 7fb85f7c7640 1 --2- 172.21.15.69:0/2784756953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb85815fd70 0x7fb858160180 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:07.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.583+0000 7fb85f7c7640 1 --2- 172.21.15.69:0/2784756953 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8580cdaa0 0x7fb8581f46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:07.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.583+0000 7fb85f7c7640 1 -- 172.21.15.69:0/2784756953 >> 172.21.15.69:0/2784756953 conn(0x7fb85808d110 msgr2=0x7fb8580684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:07.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.584+0000 7fb85f7c7640 1 -- 172.21.15.69:0/2784756953 shutdown_connections 2024-04-25T11:00:07.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.584+0000 7fb85f7c7640 1 -- 172.21.15.69:0/2784756953 wait complete. 2024-04-25T11:00:07.595 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:00:07.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.702+0000 7efcaf5eb640 1 Processor -- start 2024-04-25T11:00:07.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.703+0000 7efcaf5eb640 1 -- start start 2024-04-25T11:00:07.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.704+0000 7efcaf5eb640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7efca80b4830 0x7efca806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:07.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.704+0000 7efcaf5eb640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efca80b2dc0 0x7efca80b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:07.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.704+0000 7efcaf5eb640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efca806d870 con 0x7efca80d4570 2024-04-25T11:00:07.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.704+0000 7efcaf5eb640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7efca80b4c40 con 0x7efca80b4830 2024-04-25T11:00:07.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.704+0000 7efcaf5eb640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7efca806e460 con 0x7efca80b2dc0 2024-04-25T11:00:07.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.704+0000 7efcadb88640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efca80b2dc0 0x7efca80b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:07.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.704+0000 7efcacb86640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7efca80b4830 0x7efca806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:07.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.704+0000 7efcacb86640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7efca80b4830 0x7efca806e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.69:36368/0 (socket says 172.21.15.69:36368) 2024-04-25T11:00:07.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.704+0000 7efcadb88640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efca80b2dc0 0x7efca80b31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.69:60392/0 (socket says 172.21.15.69:60392) 2024-04-25T11:00:07.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.704+0000 7efcacb86640 1 -- 172.21.15.69:0/1744433172 learned_addr learned my addr 172.21.15.69:0/1744433172 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:07.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.704+0000 7efcad387640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7efca80d4570 0x7efca80d04b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.69:51522/0 (socket says 172.21.15.69:51522) 2024-04-25T11:00:07.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.705+0000 7efcadb88640 1 -- 172.21.15.69:0/1744433172 >> v2:172.21.15.146:3300/0 conn(0x7efca80b4830 msgr2=0x7efca806e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:07.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.705+0000 7efcadb88640 1 --2- 172.21.15.69:0/1744433172 >> v2:172.21.15.146:3300/0 conn(0x7efca80b4830 0x7efca806e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:07.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.705+0000 7efcadb88640 1 -- 172.21.15.69:0/1744433172 >> v1:172.21.15.146:6789/0 conn(0x7efca80d4570 legacy=0x7efca80d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:07.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.705+0000 7efc9ffff640 1 -- 172.21.15.69:0/1744433172 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2743102275 0 0) 0x7efca806d870 con 0x7efca80d4570 2024-04-25T11:00:07.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.705+0000 7efcadb88640 1 -- 172.21.15.69:0/1744433172 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca806db10 con 0x7efca80b2dc0 2024-04-25T11:00:07.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.706+0000 7efcadb88640 1 --2- 172.21.15.69:0/1744433172 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efca80b2dc0 0x7efca80b31d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efca806e5e0 tx=0x7efca002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57552645d134426a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:07.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.706+0000 7efc9ffff640 1 -- 172.21.15.69:0/1744433172 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca0037070 con 0x7efca80b2dc0 2024-04-25T11:00:07.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.706+0000 7efc9ffff640 1 -- 172.21.15.69:0/1744433172 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efca002ae50 con 0x7efca80b2dc0 2024-04-25T11:00:07.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.706+0000 7efc9ffff640 1 -- 172.21.15.69:0/1744433172 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca0032660 con 0x7efca80b2dc0 2024-04-25T11:00:07.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.706+0000 7efcaf5eb640 1 -- 172.21.15.69:0/1744433172 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efca80b2dc0 msgr2=0x7efca80b31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:07.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.706+0000 7efcaf5eb640 1 --2- 172.21.15.69:0/1744433172 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efca80b2dc0 0x7efca80b31d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efca806e5e0 tx=0x7efca002a100 comp rx=0 tx=0).stop 2024-04-25T11:00:07.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.707+0000 7efcaf5eb640 1 -- 172.21.15.69:0/1744433172 shutdown_connections 2024-04-25T11:00:07.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.707+0000 7efcaf5eb640 1 --2- 172.21.15.69:0/1744433172 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efca80b2dc0 0x7efca80b31d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:07.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.707+0000 7efcaf5eb640 1 --2- 172.21.15.69:0/1744433172 >> v2:172.21.15.146:3300/0 conn(0x7efca80b4830 0x7efca806e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:07.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.707+0000 7efcaf5eb640 1 -- 172.21.15.69:0/1744433172 >> 172.21.15.69:0/1744433172 conn(0x7efca80d3a50 msgr2=0x7efca80d3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:07.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.707+0000 7efcaf5eb640 1 -- 172.21.15.69:0/1744433172 shutdown_connections 2024-04-25T11:00:07.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.707+0000 7efcaf5eb640 1 -- 172.21.15.69:0/1744433172 wait complete. 2024-04-25T11:00:07.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.708+0000 7efcaf5eb640 1 Processor -- start 2024-04-25T11:00:07.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.708+0000 7efcaf5eb640 1 -- start start 2024-04-25T11:00:07.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.710+0000 7efcaf5eb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efca80b2dc0 0x7efca814ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:07.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.710+0000 7efcaf5eb640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efca80b4830 0x7efca815f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:07.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.710+0000 7efcaf5eb640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efca80d4570 0x7efca814b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:07.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.710+0000 7efcaf5eb640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efca80d0fb0 con 0x7efca80b2dc0 2024-04-25T11:00:07.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.710+0000 7efcaf5eb640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7efca80d0bc0 con 0x7efca80b4830 2024-04-25T11:00:07.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.710+0000 7efcaf5eb640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7efca80b3710 con 0x7efca80d4570 2024-04-25T11:00:07.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.710+0000 7efcad387640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efca80b2dc0 0x7efca814ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:07.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.710+0000 7efcad387640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efca80b2dc0 0x7efca814ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36488/0 (socket says 172.21.15.69:36488) 2024-04-25T11:00:07.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.710+0000 7efcad387640 1 -- 172.21.15.69:0/903469122 learned_addr learned my addr 172.21.15.69:0/903469122 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:07.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.710+0000 7efcadb88640 1 --2- 172.21.15.69:0/903469122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efca80d4570 0x7efca814b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:07.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.710+0000 7efcacb86640 1 --2- 172.21.15.69:0/903469122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efca80b4830 0x7efca815f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:07.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.710+0000 7efcad387640 1 -- 172.21.15.69:0/903469122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efca80d4570 msgr2=0x7efca814b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:07.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.710+0000 7efcad387640 1 --2- 172.21.15.69:0/903469122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efca80d4570 0x7efca814b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:07.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.710+0000 7efcad387640 1 -- 172.21.15.69:0/903469122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efca80b4830 msgr2=0x7efca815f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:07.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.710+0000 7efcad387640 1 --2- 172.21.15.69:0/903469122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efca80b4830 0x7efca815f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:07.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.710+0000 7efcad387640 1 -- 172.21.15.69:0/903469122 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca814bfd0 con 0x7efca80b2dc0 2024-04-25T11:00:07.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.710+0000 7efcad387640 1 --2- 172.21.15.69:0/903469122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efca80b2dc0 0x7efca814ec00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efca400c7b0 tx=0x7efca400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:07.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.711+0000 7efc9dffb640 1 -- 172.21.15.69:0/903469122 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca4018070 con 0x7efca80b2dc0 2024-04-25T11:00:07.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.711+0000 7efc9dffb640 1 -- 172.21.15.69:0/903469122 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efca400a5e0 con 0x7efca80b2dc0 2024-04-25T11:00:07.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.711+0000 7efcaf5eb640 1 -- 172.21.15.69:0/903469122 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca80b0b20 con 0x7efca80b2dc0 2024-04-25T11:00:07.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.711+0000 7efc9dffb640 1 -- 172.21.15.69:0/903469122 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca400a9b0 con 0x7efca80b2dc0 2024-04-25T11:00:07.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.711+0000 7efcaf5eb640 1 -- 172.21.15.69:0/903469122 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efca80b0dc0 con 0x7efca80b2dc0 2024-04-25T11:00:07.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.712+0000 7efcaf5eb640 1 -- 172.21.15.69:0/903469122 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efca814c1a0 con 0x7efca80b2dc0 2024-04-25T11:00:07.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.713+0000 7efc9dffb640 1 -- 172.21.15.69:0/903469122 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efca401b070 con 0x7efca80b2dc0 2024-04-25T11:00:07.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.713+0000 7efc9dffb640 1 --2- 172.21.15.69:0/903469122 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7efc8c0744b0 0x7efc8c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:07.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.713+0000 7efcacb86640 1 --2- 172.21.15.69:0/903469122 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7efc8c0744b0 0x7efc8c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:07.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.713+0000 7efc9dffb640 1 -- 172.21.15.69:0/903469122 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7efca4094780 con 0x7efca80b2dc0 2024-04-25T11:00:07.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.714+0000 7efcacb86640 1 --2- 172.21.15.69:0/903469122 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7efc8c0744b0 0x7efc8c0769c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efca8163a50 tx=0x7efc98009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:07.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.718+0000 7efc9dffb640 1 -- 172.21.15.69:0/903469122 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efca405e8c0 con 0x7efca80b2dc0 2024-04-25T11:00:07.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.912+0000 7efcaf5eb640 1 -- 172.21.15.69:0/903469122 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efca8050740 con 0x7efca80b2dc0 2024-04-25T11:00:07.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.913+0000 7efc9dffb640 1 -- 172.21.15.69:0/903469122 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) ==== 74+0+16682 (secure 0 0 0) 0x7efca4062bc0 con 0x7efca80b2dc0 2024-04-25T11:00:07.914 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:00:07.914 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":30,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T11:00:06.608806+0000","last_up_change":"2024-04-25T10:59:05.546839+0000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:59:06.422674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:00:05.007593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T11:00:05.558880+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6825","nonce":1876101065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6827","nonce":1876101065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6831","nonce":1876101065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6829","nonce":1876101065}]},"public_addr":"172.21.15.69:6825/1876101065","cluster_addr":"172.21.15.69:6827/1876101065","heartbeat_back_addr":"172.21.15.69:6831/1876101065","heartbeat_front_addr":"172.21.15.69:6829/1876101065","state":["exists","up"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6801","nonce":1806438563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6803","nonce":1806438563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6807","nonce":1806438563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6805","nonce":1806438563}]},"public_addr":"172.21.15.69:6801/1806438563","cluster_addr":"172.21.15.69:6803/1806438563","heartbeat_back_addr":"172.21.15.69:6807/1806438563","heartbeat_front_addr":"172.21.15.69:6805/1806438563","state":["exists","up"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6817","nonce":1593617983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6819","nonce":1593617983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6823","nonce":1593617983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6821","nonce":1593617983}]},"public_addr":"172.21.15.69:6817/1593617983","cluster_addr":"172.21.15.69:6819/1593617983","heartbeat_back_addr":"172.21.15.69:6823/1593617983","heartbeat_front_addr":"172.21.15.69:6821/1593617983","state":["exists","up"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6809","nonce":3950661294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6811","nonce":3950661294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6815","nonce":3950661294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6813","nonce":3950661294}]},"public_addr":"172.21.15.69:6809/3950661294","cluster_addr":"172.21.15.69:6811/3950661294","heartbeat_back_addr":"172.21.15.69:6815/3950661294","heartbeat_front_addr":"172.21.15.69:6813/3950661294","state":["exists","up"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6801","nonce":2508929872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6803","nonce":2508929872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6807","nonce":2508929872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6805","nonce":2508929872}]},"public_addr":"172.21.15.146:6801/2508929872","cluster_addr":"172.21.15.146:6803/2508929872","heartbeat_back_addr":"172.21.15.146:6807/2508929872","heartbeat_front_addr":"172.21.15.146:6805/2508929872","state":["exists","up"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6811","nonce":1310911138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6816","nonce":1310911138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6823","nonce":1310911138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6821","nonce":1310911138}]},"public_addr":"172.21.15.146:6811/1310911138","cluster_addr":"172.21.15.146:6816/1310911138","heartbeat_back_addr":"172.21.15.146:6823/1310911138","heartbeat_front_addr":"172.21.15.146:6821/1310911138","state":["exists","up"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6810","nonce":634688021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6813","nonce":634688021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6820","nonce":634688021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6817","nonce":634688021}]},"public_addr":"172.21.15.146:6810/634688021","cluster_addr":"172.21.15.146:6813/634688021","heartbeat_back_addr":"172.21.15.146:6820/634688021","heartbeat_front_addr":"172.21.15.146:6817/634688021","state":["exists","up"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6825","nonce":3814215935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6827","nonce":3814215935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6831","nonce":3814215935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6829","nonce":3814215935}]},"public_addr":"172.21.15.146:6825/3814215935","cluster_addr":"172.21.15.146:6827/3814215935","heartbeat_back_addr":"172.21.15.146:6831/3814215935","heartbeat_front_addr":"172.21.15.146:6829/3814215935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.820200+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.850189+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.764560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.866248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.807897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.869881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.782310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.827131+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:00:07.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.915+0000 7efcaf5eb640 1 -- 172.21.15.69:0/903469122 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7efc8c0744b0 msgr2=0x7efc8c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:07.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.915+0000 7efcaf5eb640 1 --2- 172.21.15.69:0/903469122 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7efc8c0744b0 0x7efc8c0769c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efca8163a50 tx=0x7efc98009240 comp rx=0 tx=0).stop 2024-04-25T11:00:07.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.915+0000 7efcaf5eb640 1 -- 172.21.15.69:0/903469122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efca80b2dc0 msgr2=0x7efca814ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:07.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.915+0000 7efcaf5eb640 1 --2- 172.21.15.69:0/903469122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efca80b2dc0 0x7efca814ec00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efca400c7b0 tx=0x7efca400cc70 comp rx=0 tx=0).stop 2024-04-25T11:00:07.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.915+0000 7efcaf5eb640 1 -- 172.21.15.69:0/903469122 shutdown_connections 2024-04-25T11:00:07.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.915+0000 7efcaf5eb640 1 --2- 172.21.15.69:0/903469122 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7efc8c0744b0 0x7efc8c0769c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:07.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.915+0000 7efcaf5eb640 1 --2- 172.21.15.69:0/903469122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efca80d4570 0x7efca814b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:07.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.915+0000 7efcaf5eb640 1 --2- 172.21.15.69:0/903469122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efca80b4830 0x7efca815f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:07.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.915+0000 7efcaf5eb640 1 --2- 172.21.15.69:0/903469122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efca80b2dc0 0x7efca814ec00 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:07.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.915+0000 7efcaf5eb640 1 -- 172.21.15.69:0/903469122 >> 172.21.15.69:0/903469122 conn(0x7efca80d3a50 msgr2=0x7efca80727f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:07.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.915+0000 7efcaf5eb640 1 -- 172.21.15.69:0/903469122 shutdown_connections 2024-04-25T11:00:07.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:07.916+0000 7efcaf5eb640 1 -- 172.21.15.69:0/903469122 wait complete. 2024-04-25T11:00:07.927 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T11:00:08.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.037+0000 7fc277a49640 1 Processor -- start 2024-04-25T11:00:08.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.037+0000 7fc277a49640 1 -- start start 2024-04-25T11:00:08.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.038+0000 7fc277a49640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc27008d630 0x7fc2700b90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:08.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.038+0000 7fc277a49640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc2700b1c50 0x7fc2700ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:08.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.038+0000 7fc277a49640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2700d21e0 con 0x7fc27008dec0 2024-04-25T11:00:08.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.038+0000 7fc277a49640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fc2700b95f0 con 0x7fc27008d630 2024-04-25T11:00:08.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.038+0000 7fc277a49640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc2700d1940 con 0x7fc2700b1c50 2024-04-25T11:00:08.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.038+0000 7fc2757e5640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fc27008dec0 0x7fc27008cf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:38836/0 (socket says 172.21.15.69:38836) 2024-04-25T11:00:08.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.038+0000 7fc2757e5640 1 -- 172.21.15.69:0/932906349 learned_addr learned my addr 172.21.15.69:0/932906349 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:08.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.039+0000 7fc267fff640 1 -- 172.21.15.69:0/932906349 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 13687587 0 0) 0x7fc2700d21e0 con 0x7fc27008dec0 2024-04-25T11:00:08.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.039+0000 7fc267fff640 1 -- 172.21.15.69:0/932906349 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc25c003520 con 0x7fc27008dec0 2024-04-25T11:00:08.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.039+0000 7fc274fe4640 1 --2- 172.21.15.69:0/932906349 >> v2:172.21.15.146:3300/0 conn(0x7fc27008d630 0x7fc2700b90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:08.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.039+0000 7fc275fe6640 1 --2- 172.21.15.69:0/932906349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc2700b1c50 0x7fc2700ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:08.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.039+0000 7fc267fff640 1 -- 172.21.15.69:0/932906349 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 781738789 0 0) 0x7fc25c003520 con 0x7fc27008dec0 2024-04-25T11:00:08.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.039+0000 7fc267fff640 1 -- 172.21.15.69:0/932906349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc2700b1c50 msgr2=0x7fc2700ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:08.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.039+0000 7fc267fff640 1 --2- 172.21.15.69:0/932906349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc2700b1c50 0x7fc2700ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:08.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.039+0000 7fc267fff640 1 -- 172.21.15.69:0/932906349 >> v2:172.21.15.146:3300/0 conn(0x7fc27008d630 msgr2=0x7fc2700b90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:08.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.039+0000 7fc267fff640 1 --2- 172.21.15.69:0/932906349 >> v2:172.21.15.146:3300/0 conn(0x7fc27008d630 0x7fc2700b90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:08.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.039+0000 7fc267fff640 1 -- 172.21.15.69:0/932906349 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2700d1c20 con 0x7fc27008dec0 2024-04-25T11:00:08.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.039+0000 7fc267fff640 1 -- 172.21.15.69:0/932906349 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7fc26c001a10 con 0x7fc27008dec0 2024-04-25T11:00:08.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.039+0000 7fc267fff640 1 -- 172.21.15.69:0/932906349 >> v1:172.21.15.69:6789/0 conn(0x7fc27008dec0 legacy=0x7fc27008cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:08.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.039+0000 7fc267fff640 1 --2- 172.21.15.69:0/932906349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc2700b1c50 0x7fc25c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:08.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.039+0000 7fc267fff640 1 --2- 172.21.15.69:0/932906349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc25c003f30 0x7fc25c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:08.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.039+0000 7fc267fff640 1 --2- 172.21.15.69:0/932906349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc25c004d40 0x7fc25c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:08.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.039+0000 7fc267fff640 1 -- 172.21.15.69:0/932906349 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc2700b95f0 con 0x7fc25c004d40 2024-04-25T11:00:08.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.039+0000 7fc267fff640 1 -- 172.21.15.69:0/932906349 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc2700d1940 con 0x7fc25c003f30 2024-04-25T11:00:08.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.039+0000 7fc267fff640 1 -- 172.21.15.69:0/932906349 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc25c00ba10 con 0x7fc2700b1c50 2024-04-25T11:00:08.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.039+0000 7fc275fe6640 1 --2- 172.21.15.69:0/932906349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc25c004d40 0x7fc25c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:08.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.039+0000 7fc2757e5640 1 --2- 172.21.15.69:0/932906349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc2700b1c50 0x7fc25c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:08.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.039+0000 7fc274fe4640 1 --2- 172.21.15.69:0/932906349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc25c003f30 0x7fc25c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:08.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.039+0000 7fc275fe6640 1 -- 172.21.15.69:0/932906349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc2700b1c50 msgr2=0x7fc25c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:08.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.039+0000 7fc275fe6640 1 --2- 172.21.15.69:0/932906349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc2700b1c50 0x7fc25c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:08.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.039+0000 7fc275fe6640 1 -- 172.21.15.69:0/932906349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc25c003f30 msgr2=0x7fc25c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:08.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.039+0000 7fc275fe6640 1 --2- 172.21.15.69:0/932906349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc25c003f30 0x7fc25c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:08.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.039+0000 7fc275fe6640 1 -- 172.21.15.69:0/932906349 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc25c003520 con 0x7fc25c004d40 2024-04-25T11:00:08.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.040+0000 7fc275fe6640 1 --2- 172.21.15.69:0/932906349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc25c004d40 0x7fc25c005150 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc270074270 tx=0x7fc25c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a8f9dfd2b690fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:08.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.040+0000 7fc267fff640 1 -- 172.21.15.69:0/932906349 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc268036070 con 0x7fc25c004d40 2024-04-25T11:00:08.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.040+0000 7fc267fff640 1 -- 172.21.15.69:0/932906349 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc268031400 con 0x7fc25c004d40 2024-04-25T11:00:08.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.040+0000 7fc267fff640 1 -- 172.21.15.69:0/932906349 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc268009690 con 0x7fc25c004d40 2024-04-25T11:00:08.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.041+0000 7fc277a49640 1 -- 172.21.15.69:0/932906349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc25c004d40 msgr2=0x7fc25c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:08.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.041+0000 7fc277a49640 1 --2- 172.21.15.69:0/932906349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc25c004d40 0x7fc25c005150 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc270074270 tx=0x7fc25c005830 comp rx=0 tx=0).stop 2024-04-25T11:00:08.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.041+0000 7fc275fe6640 1 -- 172.21.15.69:0/932906349 reap_dead start 2024-04-25T11:00:08.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.041+0000 7fc277a49640 1 -- 172.21.15.69:0/932906349 shutdown_connections 2024-04-25T11:00:08.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.041+0000 7fc277a49640 1 -- 172.21.15.69:0/932906349 >> 172.21.15.69:0/932906349 conn(0x7fc2700d0510 msgr2=0x7fc2700d0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:08.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.041+0000 7fc277a49640 1 -- 172.21.15.69:0/932906349 shutdown_connections 2024-04-25T11:00:08.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.041+0000 7fc277a49640 1 -- 172.21.15.69:0/932906349 wait complete. 2024-04-25T11:00:08.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.043+0000 7fc277a49640 1 Processor -- start 2024-04-25T11:00:08.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.043+0000 7fc277a49640 1 -- start start 2024-04-25T11:00:08.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.044+0000 7fc277a49640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc25c004d40 0x7fc27015e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:08.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.044+0000 7fc277a49640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2701f6910 0x7fc27015d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:08.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.044+0000 7fc277a49640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2700c62b0 0x7fc2700c66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:08.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.044+0000 7fc277a49640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc2700d15d0 con 0x7fc2700c62b0 2024-04-25T11:00:08.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.044+0000 7fc277a49640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc2700b0720 con 0x7fc2701f6910 2024-04-25T11:00:08.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.044+0000 7fc277a49640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc2700ce880 con 0x7fc25c004d40 2024-04-25T11:00:08.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.044+0000 7fc275fe6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2700c62b0 0x7fc2700c66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:08.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.044+0000 7fc275fe6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2700c62b0 0x7fc2700c66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36512/0 (socket says 172.21.15.69:36512) 2024-04-25T11:00:08.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.044+0000 7fc275fe6640 1 -- 172.21.15.69:0/3464941706 learned_addr learned my addr 172.21.15.69:0/3464941706 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:08.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.044+0000 7fc274fe4640 1 --2- 172.21.15.69:0/3464941706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2701f6910 0x7fc27015d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:08.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.044+0000 7fc2757e5640 1 --2- 172.21.15.69:0/3464941706 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc25c004d40 0x7fc27015e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:08.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.044+0000 7fc275fe6640 1 -- 172.21.15.69:0/3464941706 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc25c004d40 msgr2=0x7fc27015e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:08.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.044+0000 7fc275fe6640 1 --2- 172.21.15.69:0/3464941706 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc25c004d40 0x7fc27015e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:08.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.044+0000 7fc275fe6640 1 -- 172.21.15.69:0/3464941706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2701f6910 msgr2=0x7fc27015d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:08.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.044+0000 7fc275fe6640 1 --2- 172.21.15.69:0/3464941706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2701f6910 0x7fc27015d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:08.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.045+0000 7fc275fe6640 1 -- 172.21.15.69:0/3464941706 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2700c6c00 con 0x7fc2700c62b0 2024-04-25T11:00:08.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.045+0000 7fc275fe6640 1 --2- 172.21.15.69:0/3464941706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2700c62b0 0x7fc2700c66c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc268009b80 tx=0x7fc268004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:08.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.045+0000 7fc265ffb640 1 -- 172.21.15.69:0/3464941706 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc268036040 con 0x7fc2700c62b0 2024-04-25T11:00:08.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.045+0000 7fc265ffb640 1 -- 172.21.15.69:0/3464941706 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc268041070 con 0x7fc2700c62b0 2024-04-25T11:00:08.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.045+0000 7fc277a49640 1 -- 172.21.15.69:0/3464941706 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc27015f150 con 0x7fc2700c62b0 2024-04-25T11:00:08.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.045+0000 7fc265ffb640 1 -- 172.21.15.69:0/3464941706 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc268009690 con 0x7fc2700c62b0 2024-04-25T11:00:08.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.045+0000 7fc277a49640 1 -- 172.21.15.69:0/3464941706 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc27015f710 con 0x7fc2700c62b0 2024-04-25T11:00:08.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.046+0000 7fc277a49640 1 -- 172.21.15.69:0/3464941706 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc27015eec0 con 0x7fc2700c62b0 2024-04-25T11:00:08.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.047+0000 7fc265ffb640 1 -- 172.21.15.69:0/3464941706 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc268004270 con 0x7fc2700c62b0 2024-04-25T11:00:08.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.047+0000 7fc265ffb640 1 --2- 172.21.15.69:0/3464941706 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc2540744b0 0x7fc2540769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:08.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.047+0000 7fc2757e5640 1 --2- 172.21.15.69:0/3464941706 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc2540744b0 0x7fc2540769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:08.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.047+0000 7fc265ffb640 1 -- 172.21.15.69:0/3464941706 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7fc2680b4d40 con 0x7fc2700c62b0 2024-04-25T11:00:08.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.048+0000 7fc2757e5640 1 --2- 172.21.15.69:0/3464941706 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc2540744b0 0x7fc2540769c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc2700749e0 tx=0x7fc26c006040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:08.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.052+0000 7fc265ffb640 1 -- 172.21.15.69:0/3464941706 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc26804a020 con 0x7fc2700c62b0 2024-04-25T11:00:08.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:08.267+0000 7fc277a49640 1 -- 172.21.15.69:0/3464941706 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc270050740 con 0x7fc2700c62b0 2024-04-25T11:00:09.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.262+0000 7fc265ffb640 1 -- 172.21.15.69:0/3464941706 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) ==== 118+0+0 (secure 0 0 0) 0x7fc2700c6c00 con 0x7fc2700c62b0 2024-04-25T11:00:09.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.265+0000 7fc277a49640 1 -- 172.21.15.69:0/3464941706 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc2540744b0 msgr2=0x7fc2540769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:09.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.265+0000 7fc277a49640 1 --2- 172.21.15.69:0/3464941706 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc2540744b0 0x7fc2540769c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc2700749e0 tx=0x7fc26c006040 comp rx=0 tx=0).stop 2024-04-25T11:00:09.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.265+0000 7fc277a49640 1 -- 172.21.15.69:0/3464941706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2700c62b0 msgr2=0x7fc2700c66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:09.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.265+0000 7fc277a49640 1 --2- 172.21.15.69:0/3464941706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2700c62b0 0x7fc2700c66c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc268009b80 tx=0x7fc268004730 comp rx=0 tx=0).stop 2024-04-25T11:00:09.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.265+0000 7fc277a49640 1 -- 172.21.15.69:0/3464941706 shutdown_connections 2024-04-25T11:00:09.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.265+0000 7fc277a49640 1 --2- 172.21.15.69:0/3464941706 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc2540744b0 0x7fc2540769c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:09.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.265+0000 7fc277a49640 1 --2- 172.21.15.69:0/3464941706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2700c62b0 0x7fc2700c66c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:09.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.265+0000 7fc277a49640 1 --2- 172.21.15.69:0/3464941706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2701f6910 0x7fc27015d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:09.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.265+0000 7fc277a49640 1 --2- 172.21.15.69:0/3464941706 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc25c004d40 0x7fc27015e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:09.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.265+0000 7fc277a49640 1 -- 172.21.15.69:0/3464941706 >> 172.21.15.69:0/3464941706 conn(0x7fc2700d0510 msgr2=0x7fc2700b7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:09.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.265+0000 7fc277a49640 1 -- 172.21.15.69:0/3464941706 shutdown_connections 2024-04-25T11:00:09.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.265+0000 7fc277a49640 1 -- 172.21.15.69:0/3464941706 wait complete. 2024-04-25T11:00:09.278 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T11:00:09.278 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T11:00:09.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.393+0000 7fcdf4e20640 1 Processor -- start 2024-04-25T11:00:09.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.393+0000 7fcdf4e20640 1 -- start start 2024-04-25T11:00:09.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.394+0000 7fcdf4e20640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fcdf0161710 0x7fcdf015e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:09.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.394+0000 7fcdf4e20640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcdf006c5c0 0x7fcdf006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:09.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.394+0000 7fcdf4e20640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdf01609b0 con 0x7fcdf0163fa0 2024-04-25T11:00:09.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.394+0000 7fcdf4e20640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fcdf006cf10 con 0x7fcdf0161710 2024-04-25T11:00:09.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.394+0000 7fcdf4e20640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fcdf0164cd0 con 0x7fcdf006c5c0 2024-04-25T11:00:09.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.394+0000 7fcdee59c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fcdf0161710 0x7fcdf015e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:09.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.394+0000 7fcdee59c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fcdf0161710 0x7fcdf015e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36526/0 (socket says 172.21.15.69:36526) 2024-04-25T11:00:09.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.394+0000 7fcdee59c640 1 -- 172.21.15.69:0/3550204269 learned_addr learned my addr 172.21.15.69:0/3550204269 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:09.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.395+0000 7fcdee59c640 1 -- 172.21.15.69:0/3550204269 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcdf006c5c0 msgr2=0x7fcdf006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:09.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.395+0000 7fcdee59c640 1 --2- 172.21.15.69:0/3550204269 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcdf006c5c0 0x7fcdf006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:09.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.395+0000 7fcdee59c640 1 -- 172.21.15.69:0/3550204269 >> v1:172.21.15.146:6789/0 conn(0x7fcdf0163fa0 legacy=0x7fcdf01643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:09.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.395+0000 7fcdee59c640 1 -- 172.21.15.69:0/3550204269 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf0160c30 con 0x7fcdf0161710 2024-04-25T11:00:09.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.395+0000 7fcdee59c640 1 --2- 172.21.15.69:0/3550204269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdf0161710 0x7fcdf015e080 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcde8009e50 tx=0x7fcde802a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b49b638fbb3d71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:09.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.395+0000 7fcded59a640 1 -- 172.21.15.69:0/3550204269 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde8036070 con 0x7fcdf0161710 2024-04-25T11:00:09.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.395+0000 7fcded59a640 1 -- 172.21.15.69:0/3550204269 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcde8031600 con 0x7fcdf0161710 2024-04-25T11:00:09.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.395+0000 7fcded59a640 1 -- 172.21.15.69:0/3550204269 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde80319d0 con 0x7fcdf0161710 2024-04-25T11:00:09.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.396+0000 7fcdf4e20640 1 -- 172.21.15.69:0/3550204269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdf0161710 msgr2=0x7fcdf015e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:09.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.396+0000 7fcdf4e20640 1 --2- 172.21.15.69:0/3550204269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdf0161710 0x7fcdf015e080 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcde8009e50 tx=0x7fcde802a630 comp rx=0 tx=0).stop 2024-04-25T11:00:09.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.396+0000 7fcdf4e20640 1 -- 172.21.15.69:0/3550204269 shutdown_connections 2024-04-25T11:00:09.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.396+0000 7fcdf4e20640 1 --2- 172.21.15.69:0/3550204269 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcdf006c5c0 0x7fcdf006c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:09.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.396+0000 7fcdf4e20640 1 --2- 172.21.15.69:0/3550204269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdf0161710 0x7fcdf015e080 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:09.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.396+0000 7fcdf4e20640 1 -- 172.21.15.69:0/3550204269 >> 172.21.15.69:0/3550204269 conn(0x7fcdf00d5d90 msgr2=0x7fcdf00d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:09.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.396+0000 7fcdf4e20640 1 -- 172.21.15.69:0/3550204269 shutdown_connections 2024-04-25T11:00:09.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.396+0000 7fcdf4e20640 1 -- 172.21.15.69:0/3550204269 wait complete. 2024-04-25T11:00:09.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.398+0000 7fcdf4e20640 1 Processor -- start 2024-04-25T11:00:09.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.398+0000 7fcdf4e20640 1 -- start start 2024-04-25T11:00:09.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.399+0000 7fcdf4e20640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcdf006c5c0 0x7fcdf00ba5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:09.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.399+0000 7fcdf4e20640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdf0161710 0x7fcdf0150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:09.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.399+0000 7fcdf4e20640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcdf0163fa0 0x7fcdf0151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:09.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.399+0000 7fcdf4e20640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf00d49f0 con 0x7fcdf0161710 2024-04-25T11:00:09.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.399+0000 7fcdf4e20640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf01611f0 con 0x7fcdf006c5c0 2024-04-25T11:00:09.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.399+0000 7fcdf4e20640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fcdf00d4b70 con 0x7fcdf0163fa0 2024-04-25T11:00:09.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.400+0000 7fcdedd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdf0161710 0x7fcdf0150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:09.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.400+0000 7fcdedd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdf0161710 0x7fcdf0150fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36538/0 (socket says 172.21.15.69:36538) 2024-04-25T11:00:09.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.400+0000 7fcdedd9b640 1 -- 172.21.15.69:0/2677246730 learned_addr learned my addr 172.21.15.69:0/2677246730 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:09.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.400+0000 7fcdee59c640 1 --2- 172.21.15.69:0/2677246730 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcdf006c5c0 0x7fcdf00ba5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:09.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.400+0000 7fcdeed9d640 1 --2- 172.21.15.69:0/2677246730 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcdf0163fa0 0x7fcdf0151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:09.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.400+0000 7fcdedd9b640 1 -- 172.21.15.69:0/2677246730 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcdf0163fa0 msgr2=0x7fcdf0151520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:09.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.400+0000 7fcdedd9b640 1 --2- 172.21.15.69:0/2677246730 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcdf0163fa0 0x7fcdf0151520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:09.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.400+0000 7fcdedd9b640 1 -- 172.21.15.69:0/2677246730 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcdf006c5c0 msgr2=0x7fcdf00ba5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:09.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.400+0000 7fcdedd9b640 1 --2- 172.21.15.69:0/2677246730 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcdf006c5c0 0x7fcdf00ba5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:09.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.400+0000 7fcdedd9b640 1 -- 172.21.15.69:0/2677246730 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf01f6fc0 con 0x7fcdf0161710 2024-04-25T11:00:09.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.400+0000 7fcdedd9b640 1 --2- 172.21.15.69:0/2677246730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdf0161710 0x7fcdf0150fe0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcdd800b410 tx=0x7fcdd800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:09.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.400+0000 7fcde6ffd640 1 -- 172.21.15.69:0/2677246730 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8015020 con 0x7fcdf0161710 2024-04-25T11:00:09.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.400+0000 7fcde6ffd640 1 -- 172.21.15.69:0/2677246730 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdd8011850 con 0x7fcdf0161710 2024-04-25T11:00:09.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.400+0000 7fcde6ffd640 1 -- 172.21.15.69:0/2677246730 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8011c40 con 0x7fcdf0161710 2024-04-25T11:00:09.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.400+0000 7fcdf4e20640 1 -- 172.21.15.69:0/2677246730 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdf00cc230 con 0x7fcdf0161710 2024-04-25T11:00:09.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.400+0000 7fcdf4e20640 1 -- 172.21.15.69:0/2677246730 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdf01f7540 con 0x7fcdf0161710 2024-04-25T11:00:09.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.401+0000 7fcdf4e20640 1 -- 172.21.15.69:0/2677246730 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdf00d3540 con 0x7fcdf0161710 2024-04-25T11:00:09.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.402+0000 7fcde6ffd640 1 -- 172.21.15.69:0/2677246730 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcdd8018020 con 0x7fcdf0161710 2024-04-25T11:00:09.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.402+0000 7fcde6ffd640 1 --2- 172.21.15.69:0/2677246730 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fcdd00744b0 0x7fcdd00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:09.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.403+0000 7fcde6ffd640 1 -- 172.21.15.69:0/2677246730 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fcdd8095010 con 0x7fcdf0161710 2024-04-25T11:00:09.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.403+0000 7fcdee59c640 1 --2- 172.21.15.69:0/2677246730 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fcdd00744b0 0x7fcdd00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:09.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.403+0000 7fcdee59c640 1 --2- 172.21.15.69:0/2677246730 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fcdd00744b0 0x7fcdd00769c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcde8033000 tx=0x7fcde8009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:09.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.407+0000 7fcde6ffd640 1 -- 172.21.15.69:0/2677246730 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcdf00d3540 con 0x7fcdf0161710 2024-04-25T11:00:09.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:09.620+0000 7fcdf4e20640 1 -- 172.21.15.69:0/2677246730 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fcdf00cc460 con 0x7fcdf0161710 2024-04-25T11:00:10.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.267+0000 7fcde6ffd640 1 -- 172.21.15.69:0/2677246730 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) ==== 122+0+0 (secure 0 0 0) 0x7fcdd801f020 con 0x7fcdf0161710 2024-04-25T11:00:10.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.269+0000 7fcdf4e20640 1 -- 172.21.15.69:0/2677246730 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fcdd00744b0 msgr2=0x7fcdd00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:10.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.269+0000 7fcdf4e20640 1 --2- 172.21.15.69:0/2677246730 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fcdd00744b0 0x7fcdd00769c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcde8033000 tx=0x7fcde8009000 comp rx=0 tx=0).stop 2024-04-25T11:00:10.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.269+0000 7fcdf4e20640 1 -- 172.21.15.69:0/2677246730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdf0161710 msgr2=0x7fcdf0150fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:10.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.269+0000 7fcdf4e20640 1 --2- 172.21.15.69:0/2677246730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdf0161710 0x7fcdf0150fe0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcdd800b410 tx=0x7fcdd800b8d0 comp rx=0 tx=0).stop 2024-04-25T11:00:10.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.270+0000 7fcdf4e20640 1 -- 172.21.15.69:0/2677246730 shutdown_connections 2024-04-25T11:00:10.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.270+0000 7fcdf4e20640 1 --2- 172.21.15.69:0/2677246730 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fcdd00744b0 0x7fcdd00769c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:10.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.270+0000 7fcdf4e20640 1 --2- 172.21.15.69:0/2677246730 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcdf0163fa0 0x7fcdf0151520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:10.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.270+0000 7fcdf4e20640 1 --2- 172.21.15.69:0/2677246730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdf0161710 0x7fcdf0150fe0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:10.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.270+0000 7fcdf4e20640 1 --2- 172.21.15.69:0/2677246730 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcdf006c5c0 0x7fcdf00ba5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:10.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.270+0000 7fcdf4e20640 1 -- 172.21.15.69:0/2677246730 >> 172.21.15.69:0/2677246730 conn(0x7fcdf00d5d90 msgr2=0x7fcdf015f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:10.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.270+0000 7fcdf4e20640 1 -- 172.21.15.69:0/2677246730 shutdown_connections 2024-04-25T11:00:10.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.270+0000 7fcdf4e20640 1 -- 172.21.15.69:0/2677246730 wait complete. 2024-04-25T11:00:10.282 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T11:00:10.282 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:00:10.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.391+0000 7f849c8aa640 1 Processor -- start 2024-04-25T11:00:10.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.391+0000 7f849c8aa640 1 -- start start 2024-04-25T11:00:10.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.392+0000 7f849c8aa640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f84980b2cc0 0x7f84980b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:10.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.392+0000 7f849c8aa640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84980d3860 0x7f84980d3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:10.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.392+0000 7f849c8aa640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84980d41b0 con 0x7f84980b9160 2024-04-25T11:00:10.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.392+0000 7f849c8aa640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f84980b4ab0 con 0x7f84980b2cc0 2024-04-25T11:00:10.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.392+0000 7f849c8aa640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f84980b4180 con 0x7f84980d3860 2024-04-25T11:00:10.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.393+0000 7f8496d9d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84980d3860 0x7f84980d3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:10.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.393+0000 7f849659c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f84980b2cc0 0x7f84980b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:10.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.393+0000 7f8496d9d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84980d3860 0x7f84980d3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.69:60476/0 (socket says 172.21.15.69:60476) 2024-04-25T11:00:10.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.393+0000 7f8496d9d640 1 -- 172.21.15.69:0/408224529 learned_addr learned my addr 172.21.15.69:0/408224529 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:10.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.393+0000 7f8495d9b640 1 --1- 172.21.15.69:0/408224529 >> v1:172.21.15.146:6789/0 conn(0x7f84980b9160 0x7f84980d27e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.69:51536/0 (socket says 172.21.15.69:51536) 2024-04-25T11:00:10.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.393+0000 7f8496d9d640 1 -- 172.21.15.69:0/408224529 >> v2:172.21.15.146:3300/0 conn(0x7f84980b2cc0 msgr2=0x7f84980b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:10.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.393+0000 7f8496d9d640 1 --2- 172.21.15.69:0/408224529 >> v2:172.21.15.146:3300/0 conn(0x7f84980b2cc0 0x7f84980b4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:10.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.393+0000 7f8496d9d640 1 -- 172.21.15.69:0/408224529 >> v1:172.21.15.146:6789/0 conn(0x7f84980b9160 legacy=0x7f84980d27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:10.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.393+0000 7f8496d9d640 1 -- 172.21.15.69:0/408224529 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84980b4460 con 0x7f84980d3860 2024-04-25T11:00:10.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.394+0000 7f8496d9d640 1 --2- 172.21.15.69:0/408224529 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84980d3860 0x7f84980d3c70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8488009870 tx=0x7f848802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c191ec35a0243ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:10.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.394+0000 7f849559a640 1 -- 172.21.15.69:0/408224529 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8488037070 con 0x7f84980d3860 2024-04-25T11:00:10.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.394+0000 7f849559a640 1 -- 172.21.15.69:0/408224529 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8488032440 con 0x7f84980d3860 2024-04-25T11:00:10.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.394+0000 7f849559a640 1 -- 172.21.15.69:0/408224529 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8488032810 con 0x7f84980d3860 2024-04-25T11:00:10.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.395+0000 7f849c8aa640 1 -- 172.21.15.69:0/408224529 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84980d3860 msgr2=0x7f84980d3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:10.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.395+0000 7f849c8aa640 1 --2- 172.21.15.69:0/408224529 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84980d3860 0x7f84980d3c70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8488009870 tx=0x7f848802a1c0 comp rx=0 tx=0).stop 2024-04-25T11:00:10.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.395+0000 7f849c8aa640 1 -- 172.21.15.69:0/408224529 shutdown_connections 2024-04-25T11:00:10.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.395+0000 7f849c8aa640 1 --2- 172.21.15.69:0/408224529 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84980d3860 0x7f84980d3c70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:10.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.395+0000 7f849c8aa640 1 --2- 172.21.15.69:0/408224529 >> v2:172.21.15.146:3300/0 conn(0x7f84980b2cc0 0x7f84980b4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:10.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.395+0000 7f849c8aa640 1 -- 172.21.15.69:0/408224529 >> 172.21.15.69:0/408224529 conn(0x7f849808d110 msgr2=0x7f849808d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:10.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.395+0000 7f849c8aa640 1 -- 172.21.15.69:0/408224529 shutdown_connections 2024-04-25T11:00:10.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.395+0000 7f849c8aa640 1 -- 172.21.15.69:0/408224529 wait complete. 2024-04-25T11:00:10.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.397+0000 7f849c8aa640 1 Processor -- start 2024-04-25T11:00:10.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.397+0000 7f849c8aa640 1 -- start start 2024-04-25T11:00:10.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.398+0000 7f849c8aa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84980b2cc0 0x7f849814e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:10.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.398+0000 7f849c8aa640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84980b9160 0x7f84981634d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:10.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.398+0000 7f849c8aa640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84980d3860 0x7f849815c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:10.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.398+0000 7f849c8aa640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f84980b05c0 con 0x7f84980b2cc0 2024-04-25T11:00:10.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.398+0000 7f849c8aa640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f84980b9570 con 0x7f84980b9160 2024-04-25T11:00:10.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.398+0000 7f849c8aa640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f84980b0740 con 0x7f84980d3860 2024-04-25T11:00:10.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.398+0000 7f849659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84980b2cc0 0x7f849814e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:10.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.398+0000 7f849659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84980b2cc0 0x7f849814e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36548/0 (socket says 172.21.15.69:36548) 2024-04-25T11:00:10.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.398+0000 7f849659c640 1 -- 172.21.15.69:0/492479758 learned_addr learned my addr 172.21.15.69:0/492479758 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:10.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.399+0000 7f8496d9d640 1 --2- 172.21.15.69:0/492479758 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84980d3860 0x7f849815c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:10.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.399+0000 7f8495d9b640 1 --2- 172.21.15.69:0/492479758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84980b9160 0x7f84981634d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:10.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.399+0000 7f849659c640 1 -- 172.21.15.69:0/492479758 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84980d3860 msgr2=0x7f849815c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:10.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.399+0000 7f849659c640 1 --2- 172.21.15.69:0/492479758 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84980d3860 0x7f849815c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:10.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.399+0000 7f849659c640 1 -- 172.21.15.69:0/492479758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84980b9160 msgr2=0x7f84981634d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:10.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.399+0000 7f849659c640 1 --2- 172.21.15.69:0/492479758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84980b9160 0x7f84981634d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:10.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.399+0000 7f849659c640 1 -- 172.21.15.69:0/492479758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f849815b560 con 0x7f84980b2cc0 2024-04-25T11:00:10.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.399+0000 7f849659c640 1 --2- 172.21.15.69:0/492479758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84980b2cc0 0x7f849814e1e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f849000efc0 tx=0x7f849000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:10.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.399+0000 7f8486ffd640 1 -- 172.21.15.69:0/492479758 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490018020 con 0x7f84980b2cc0 2024-04-25T11:00:10.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.399+0000 7f8486ffd640 1 -- 172.21.15.69:0/492479758 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8490014480 con 0x7f84980b2cc0 2024-04-25T11:00:10.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.399+0000 7f849c8aa640 1 -- 172.21.15.69:0/492479758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f849815b840 con 0x7f84980b2cc0 2024-04-25T11:00:10.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.399+0000 7f8486ffd640 1 -- 172.21.15.69:0/492479758 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490014850 con 0x7f84980b2cc0 2024-04-25T11:00:10.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.399+0000 7f849c8aa640 1 -- 172.21.15.69:0/492479758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f849815be00 con 0x7f84980b2cc0 2024-04-25T11:00:10.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.400+0000 7f849c8aa640 1 -- 172.21.15.69:0/492479758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8498050740 con 0x7f84980b2cc0 2024-04-25T11:00:10.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.401+0000 7f8486ffd640 1 -- 172.21.15.69:0/492479758 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f849001b020 con 0x7f84980b2cc0 2024-04-25T11:00:10.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.401+0000 7f8486ffd640 1 --2- 172.21.15.69:0/492479758 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f84780744b0 0x7f84780769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:10.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.401+0000 7f8495d9b640 1 --2- 172.21.15.69:0/492479758 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f84780744b0 0x7f84780769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:10.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.401+0000 7f8486ffd640 1 -- 172.21.15.69:0/492479758 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f8490095cb0 con 0x7f84980b2cc0 2024-04-25T11:00:10.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.402+0000 7f8495d9b640 1 --2- 172.21.15.69:0/492479758 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f84780744b0 0x7f84780769c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f84980d3650 tx=0x7f848c007ce0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:10.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.406+0000 7f8486ffd640 1 -- 172.21.15.69:0/492479758 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8490022030 con 0x7f84980b2cc0 2024-04-25T11:00:10.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.617+0000 7f849c8aa640 1 -- 172.21.15.69:0/492479758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84980b5f60 con 0x7f84980b2cc0 2024-04-25T11:00:10.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.617+0000 7f8486ffd640 1 -- 172.21.15.69:0/492479758 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5177 (secure 0 0 0) 0x7f8490010040 con 0x7f84980b2cc0 2024-04-25T11:00:10.618 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:00:10.618 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2223802951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":2223802951},{"type":"v1","addr":"172.21.15.146:6835","nonce":2223802951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4867,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/8243043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":8243043},{"type":"v1","addr":"172.21.15.146:6834","nonce":8243043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5031,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/3766456547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3766456547},{"type":"v1","addr":"172.21.15.69:6836","nonce":3766456547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:00:06.608766+0000","modified":"2024-04-25T11:00:09.621738+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5034},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5034":{"gid":5034,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6837/3153221129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":3153221129},{"type":"v1","addr":"172.21.15.69:6837","nonce":3153221129}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5034,"qdb_cluster":[5034]},"id":2}]} 2024-04-25T11:00:10.618 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 15 2024-04-25T11:00:10.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.619+0000 7f849c8aa640 1 -- 172.21.15.69:0/492479758 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f84780744b0 msgr2=0x7f84780769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:10.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.619+0000 7f849c8aa640 1 --2- 172.21.15.69:0/492479758 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f84780744b0 0x7f84780769c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f84980d3650 tx=0x7f848c007ce0 comp rx=0 tx=0).stop 2024-04-25T11:00:10.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.619+0000 7f849c8aa640 1 -- 172.21.15.69:0/492479758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84980b2cc0 msgr2=0x7f849814e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:10.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.619+0000 7f849c8aa640 1 --2- 172.21.15.69:0/492479758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84980b2cc0 0x7f849814e1e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f849000efc0 tx=0x7f849000c490 comp rx=0 tx=0).stop 2024-04-25T11:00:10.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.619+0000 7f849c8aa640 1 -- 172.21.15.69:0/492479758 shutdown_connections 2024-04-25T11:00:10.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.619+0000 7f849c8aa640 1 --2- 172.21.15.69:0/492479758 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f84780744b0 0x7f84780769c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:10.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.619+0000 7f849c8aa640 1 --2- 172.21.15.69:0/492479758 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84980d3860 0x7f849815c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:10.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.619+0000 7f849c8aa640 1 --2- 172.21.15.69:0/492479758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84980b9160 0x7f84981634d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:10.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.619+0000 7f849c8aa640 1 --2- 172.21.15.69:0/492479758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84980b2cc0 0x7f849814e1e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:10.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.619+0000 7f849c8aa640 1 -- 172.21.15.69:0/492479758 >> 172.21.15.69:0/492479758 conn(0x7f849808d110 msgr2=0x7f84980b4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:10.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.619+0000 7f849c8aa640 1 -- 172.21.15.69:0/492479758 shutdown_connections 2024-04-25T11:00:10.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.619+0000 7f849c8aa640 1 -- 172.21.15.69:0/492479758 wait complete. 2024-04-25T11:00:10.631 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:00:10.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.738+0000 7fa0423c5640 1 Processor -- start 2024-04-25T11:00:10.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.738+0000 7fa0423c5640 1 -- start start 2024-04-25T11:00:10.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.740+0000 7fa0423c5640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa03c143440 0x7fa03c160b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:10.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.740+0000 7fa0423c5640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa03c1610a0 0x7fa03c0d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:10.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.740+0000 7fa0423c5640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa03c1614b0 con 0x7fa03c0d0830 2024-04-25T11:00:10.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.740+0000 7fa0423c5640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fa03c06f4a0 con 0x7fa03c143440 2024-04-25T11:00:10.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.740+0000 7fa0423c5640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa03c070940 con 0x7fa03c1610a0 2024-04-25T11:00:10.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.740+0000 7fa040962640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fa03c0d0830 0x7fa03c15d590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:38844/0 (socket says 172.21.15.69:38844) 2024-04-25T11:00:10.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.740+0000 7fa040962640 1 -- 172.21.15.69:0/6440570 learned_addr learned my addr 172.21.15.69:0/6440570 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:10.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.740+0000 7fa03bfff640 1 --2- 172.21.15.69:0/6440570 >> v2:172.21.15.146:3300/0 conn(0x7fa03c143440 0x7fa03c160b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:10.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.740+0000 7fa03b7fe640 1 --2- 172.21.15.69:0/6440570 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa03c1610a0 0x7fa03c0d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:10.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.740+0000 7fa03affd640 1 -- 172.21.15.69:0/6440570 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3559475228 0 0) 0x7fa034001490 con 0x7fa03c0d0830 2024-04-25T11:00:10.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.740+0000 7fa03affd640 1 -- 172.21.15.69:0/6440570 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa024003520 con 0x7fa03c0d0830 2024-04-25T11:00:10.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.741+0000 7fa03affd640 1 -- 172.21.15.69:0/6440570 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1252303568 0 0) 0x7fa024003520 con 0x7fa03c0d0830 2024-04-25T11:00:10.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.741+0000 7fa03affd640 1 -- 172.21.15.69:0/6440570 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa03c1610a0 msgr2=0x7fa03c0d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:10.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.741+0000 7fa03affd640 1 --2- 172.21.15.69:0/6440570 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa03c1610a0 0x7fa03c0d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:10.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.741+0000 7fa03affd640 1 -- 172.21.15.69:0/6440570 >> v2:172.21.15.146:3300/0 conn(0x7fa03c143440 msgr2=0x7fa03c160b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:10.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.741+0000 7fa03affd640 1 --2- 172.21.15.69:0/6440570 >> v2:172.21.15.146:3300/0 conn(0x7fa03c143440 0x7fa03c160b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:10.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.741+0000 7fa03affd640 1 -- 172.21.15.69:0/6440570 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa03c143850 con 0x7fa03c0d0830 2024-04-25T11:00:10.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.741+0000 7fa03affd640 1 -- 172.21.15.69:0/6440570 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7fa034001bc0 con 0x7fa03c0d0830 2024-04-25T11:00:10.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.741+0000 7fa03affd640 1 -- 172.21.15.69:0/6440570 >> v1:172.21.15.69:6789/0 conn(0x7fa03c0d0830 legacy=0x7fa03c15d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:10.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.741+0000 7fa03affd640 1 --2- 172.21.15.69:0/6440570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0240039b0 0x7fa024003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:10.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.741+0000 7fa03affd640 1 --2- 172.21.15.69:0/6440570 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa024004800 0x7fa024004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:10.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.741+0000 7fa03affd640 1 --2- 172.21.15.69:0/6440570 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa03c1610a0 0x7fa024005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:10.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.741+0000 7fa03affd640 1 -- 172.21.15.69:0/6440570 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa03c06f4a0 con 0x7fa0240039b0 2024-04-25T11:00:10.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.741+0000 7fa03affd640 1 -- 172.21.15.69:0/6440570 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa03c070940 con 0x7fa024004800 2024-04-25T11:00:10.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.741+0000 7fa03affd640 1 -- 172.21.15.69:0/6440570 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa02400baa0 con 0x7fa03c1610a0 2024-04-25T11:00:10.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.741+0000 7fa03bfff640 1 --2- 172.21.15.69:0/6440570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0240039b0 0x7fa024003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:10.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.741+0000 7fa040962640 1 --2- 172.21.15.69:0/6440570 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa03c1610a0 0x7fa024005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:10.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.741+0000 7fa03b7fe640 1 --2- 172.21.15.69:0/6440570 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa024004800 0x7fa024004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:10.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.741+0000 7fa03bfff640 1 -- 172.21.15.69:0/6440570 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa03c1610a0 msgr2=0x7fa024005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:10.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.741+0000 7fa03bfff640 1 --2- 172.21.15.69:0/6440570 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa03c1610a0 0x7fa024005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:10.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.741+0000 7fa03bfff640 1 -- 172.21.15.69:0/6440570 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa024004800 msgr2=0x7fa024004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:10.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.741+0000 7fa03bfff640 1 --2- 172.21.15.69:0/6440570 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa024004800 0x7fa024004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:10.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.741+0000 7fa03bfff640 1 -- 172.21.15.69:0/6440570 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa024003520 con 0x7fa0240039b0 2024-04-25T11:00:10.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.742+0000 7fa03bfff640 1 --2- 172.21.15.69:0/6440570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0240039b0 0x7fa024003e20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa02c009a90 tx=0x7fa024005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce1f731fad069bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:10.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.742+0000 7fa03affd640 1 -- 172.21.15.69:0/6440570 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c036070 con 0x7fa0240039b0 2024-04-25T11:00:10.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.742+0000 7fa03affd640 1 -- 172.21.15.69:0/6440570 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa02c031400 con 0x7fa0240039b0 2024-04-25T11:00:10.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.742+0000 7fa03affd640 1 -- 172.21.15.69:0/6440570 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c009690 con 0x7fa0240039b0 2024-04-25T11:00:10.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.742+0000 7fa0423c5640 1 -- 172.21.15.69:0/6440570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0240039b0 msgr2=0x7fa024003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:10.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.742+0000 7fa0423c5640 1 --2- 172.21.15.69:0/6440570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0240039b0 0x7fa024003e20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa02c009a90 tx=0x7fa024005830 comp rx=0 tx=0).stop 2024-04-25T11:00:10.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.742+0000 7fa040962640 1 -- 172.21.15.69:0/6440570 reap_dead start 2024-04-25T11:00:10.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.742+0000 7fa0423c5640 1 -- 172.21.15.69:0/6440570 shutdown_connections 2024-04-25T11:00:10.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.742+0000 7fa0423c5640 1 -- 172.21.15.69:0/6440570 >> 172.21.15.69:0/6440570 conn(0x7fa03c0d1ae0 msgr2=0x7fa03c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:10.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.743+0000 7fa0423c5640 1 -- 172.21.15.69:0/6440570 shutdown_connections 2024-04-25T11:00:10.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.743+0000 7fa0423c5640 1 -- 172.21.15.69:0/6440570 wait complete. 2024-04-25T11:00:10.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.744+0000 7fa0423c5640 1 Processor -- start 2024-04-25T11:00:10.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.744+0000 7fa0423c5640 1 -- start start 2024-04-25T11:00:10.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.746+0000 7fa0423c5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0240039b0 0x7fa03c0cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:10.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.746+0000 7fa0423c5640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa03c0c7dd0 0x7fa03c142980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:10.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.746+0000 7fa0423c5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa03c0c7680 0x7fa03c1f64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:10.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.746+0000 7fa0423c5640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa03c0d1400 con 0x7fa03c0c7680 2024-04-25T11:00:10.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.746+0000 7fa0423c5640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa03c0d1120 con 0x7fa0240039b0 2024-04-25T11:00:10.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.746+0000 7fa0423c5640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa03c06f130 con 0x7fa03c0c7dd0 2024-04-25T11:00:10.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.746+0000 7fa040962640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa03c0c7680 0x7fa03c1f64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:10.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.746+0000 7fa040962640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa03c0c7680 0x7fa03c1f64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36554/0 (socket says 172.21.15.69:36554) 2024-04-25T11:00:10.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.746+0000 7fa040962640 1 -- 172.21.15.69:0/985177504 learned_addr learned my addr 172.21.15.69:0/985177504 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:10.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.746+0000 7fa03b7fe640 1 --2- 172.21.15.69:0/985177504 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa03c0c7dd0 0x7fa03c142980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:10.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.746+0000 7fa03bfff640 1 --2- 172.21.15.69:0/985177504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0240039b0 0x7fa03c0cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:10.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.746+0000 7fa040962640 1 -- 172.21.15.69:0/985177504 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa03c0c7dd0 msgr2=0x7fa03c142980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:10.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.746+0000 7fa040962640 1 --2- 172.21.15.69:0/985177504 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa03c0c7dd0 0x7fa03c142980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:10.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.746+0000 7fa040962640 1 -- 172.21.15.69:0/985177504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0240039b0 msgr2=0x7fa03c0cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:10.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.746+0000 7fa040962640 1 --2- 172.21.15.69:0/985177504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0240039b0 0x7fa03c0cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:10.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.746+0000 7fa040962640 1 -- 172.21.15.69:0/985177504 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa03c0c68e0 con 0x7fa03c0c7680 2024-04-25T11:00:10.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.746+0000 7fa03bfff640 1 --2- 172.21.15.69:0/985177504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0240039b0 0x7fa03c0cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:10.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.746+0000 7fa040962640 1 --2- 172.21.15.69:0/985177504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa03c0c7680 0x7fa03c1f64a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa03c0cf8a0 tx=0x7fa03400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:10.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.746+0000 7fa038ff9640 1 -- 172.21.15.69:0/985177504 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03400ec10 con 0x7fa03c0c7680 2024-04-25T11:00:10.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.747+0000 7fa038ff9640 1 -- 172.21.15.69:0/985177504 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa034019070 con 0x7fa03c0c7680 2024-04-25T11:00:10.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.747+0000 7fa0423c5640 1 -- 172.21.15.69:0/985177504 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa03c0c8540 con 0x7fa03c0c7680 2024-04-25T11:00:10.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.747+0000 7fa038ff9640 1 -- 172.21.15.69:0/985177504 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa034013420 con 0x7fa03c0c7680 2024-04-25T11:00:10.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.747+0000 7fa0423c5640 1 -- 172.21.15.69:0/985177504 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa03c0c6e80 con 0x7fa03c0c7680 2024-04-25T11:00:10.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.747+0000 7fa0423c5640 1 -- 172.21.15.69:0/985177504 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa03c050740 con 0x7fa03c0c7680 2024-04-25T11:00:10.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.748+0000 7fa038ff9640 1 -- 172.21.15.69:0/985177504 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa0340135c0 con 0x7fa03c0c7680 2024-04-25T11:00:10.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.748+0000 7fa038ff9640 1 --2- 172.21.15.69:0/985177504 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa0280744b0 0x7fa0280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:10.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.749+0000 7fa03bfff640 1 --2- 172.21.15.69:0/985177504 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa0280744b0 0x7fa0280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:10.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.749+0000 7fa038ff9640 1 -- 172.21.15.69:0/985177504 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fa034095580 con 0x7fa03c0c7680 2024-04-25T11:00:10.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.749+0000 7fa03bfff640 1 --2- 172.21.15.69:0/985177504 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa0280744b0 0x7fa0280769c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa02c033000 tx=0x7fa02c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:10.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.753+0000 7fa038ff9640 1 -- 172.21.15.69:0/985177504 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa03c1614b0 con 0x7fa03c0c7680 2024-04-25T11:00:10.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.927+0000 7fa0423c5640 1 -- 172.21.15.69:0/985177504 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa03c1617b0 con 0x7fa03c0c7680 2024-04-25T11:00:10.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.929+0000 7fa038ff9640 1 -- 172.21.15.69:0/985177504 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16669 (secure 0 0 0) 0x7fa03405fd10 con 0x7fa03c0c7680 2024-04-25T11:00:10.929 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:00:10.929 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":31,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T11:00:08.250154+0000","last_up_change":"2024-04-25T10:59:05.546839+0000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:59:06.422674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:00:05.007593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T11:00:05.558880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6825","nonce":1876101065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6827","nonce":1876101065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6831","nonce":1876101065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6829","nonce":1876101065}]},"public_addr":"172.21.15.69:6825/1876101065","cluster_addr":"172.21.15.69:6827/1876101065","heartbeat_back_addr":"172.21.15.69:6831/1876101065","heartbeat_front_addr":"172.21.15.69:6829/1876101065","state":["exists","up"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6801","nonce":1806438563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6803","nonce":1806438563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6807","nonce":1806438563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6805","nonce":1806438563}]},"public_addr":"172.21.15.69:6801/1806438563","cluster_addr":"172.21.15.69:6803/1806438563","heartbeat_back_addr":"172.21.15.69:6807/1806438563","heartbeat_front_addr":"172.21.15.69:6805/1806438563","state":["exists","up"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6817","nonce":1593617983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6819","nonce":1593617983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6823","nonce":1593617983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6821","nonce":1593617983}]},"public_addr":"172.21.15.69:6817/1593617983","cluster_addr":"172.21.15.69:6819/1593617983","heartbeat_back_addr":"172.21.15.69:6823/1593617983","heartbeat_front_addr":"172.21.15.69:6821/1593617983","state":["exists","up"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6809","nonce":3950661294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6811","nonce":3950661294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6815","nonce":3950661294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6813","nonce":3950661294}]},"public_addr":"172.21.15.69:6809/3950661294","cluster_addr":"172.21.15.69:6811/3950661294","heartbeat_back_addr":"172.21.15.69:6815/3950661294","heartbeat_front_addr":"172.21.15.69:6813/3950661294","state":["exists","up"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6801","nonce":2508929872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6803","nonce":2508929872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6807","nonce":2508929872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6805","nonce":2508929872}]},"public_addr":"172.21.15.146:6801/2508929872","cluster_addr":"172.21.15.146:6803/2508929872","heartbeat_back_addr":"172.21.15.146:6807/2508929872","heartbeat_front_addr":"172.21.15.146:6805/2508929872","state":["exists","up"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6811","nonce":1310911138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6816","nonce":1310911138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6823","nonce":1310911138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6821","nonce":1310911138}]},"public_addr":"172.21.15.146:6811/1310911138","cluster_addr":"172.21.15.146:6816/1310911138","heartbeat_back_addr":"172.21.15.146:6823/1310911138","heartbeat_front_addr":"172.21.15.146:6821/1310911138","state":["exists","up"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6810","nonce":634688021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6813","nonce":634688021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6820","nonce":634688021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6817","nonce":634688021}]},"public_addr":"172.21.15.146:6810/634688021","cluster_addr":"172.21.15.146:6813/634688021","heartbeat_back_addr":"172.21.15.146:6820/634688021","heartbeat_front_addr":"172.21.15.146:6817/634688021","state":["exists","up"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6825","nonce":3814215935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6827","nonce":3814215935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6831","nonce":3814215935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6829","nonce":3814215935}]},"public_addr":"172.21.15.146:6825/3814215935","cluster_addr":"172.21.15.146:6827/3814215935","heartbeat_back_addr":"172.21.15.146:6831/3814215935","heartbeat_front_addr":"172.21.15.146:6829/3814215935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.820200+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.850189+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.764560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.866248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.807897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.869881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.782310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.827131+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:00:10.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.930+0000 7fa0423c5640 1 -- 172.21.15.69:0/985177504 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa0280744b0 msgr2=0x7fa0280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:10.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.930+0000 7fa0423c5640 1 --2- 172.21.15.69:0/985177504 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa0280744b0 0x7fa0280769c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa02c033000 tx=0x7fa02c008030 comp rx=0 tx=0).stop 2024-04-25T11:00:10.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.930+0000 7fa0423c5640 1 -- 172.21.15.69:0/985177504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa03c0c7680 msgr2=0x7fa03c1f64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:10.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.930+0000 7fa0423c5640 1 --2- 172.21.15.69:0/985177504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa03c0c7680 0x7fa03c1f64a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa03c0cf8a0 tx=0x7fa03400e530 comp rx=0 tx=0).stop 2024-04-25T11:00:10.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.931+0000 7fa0423c5640 1 -- 172.21.15.69:0/985177504 shutdown_connections 2024-04-25T11:00:10.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.931+0000 7fa0423c5640 1 --2- 172.21.15.69:0/985177504 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa0280744b0 0x7fa0280769c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:10.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.931+0000 7fa0423c5640 1 --2- 172.21.15.69:0/985177504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa03c0c7680 0x7fa03c1f64a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:10.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.931+0000 7fa0423c5640 1 --2- 172.21.15.69:0/985177504 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa03c0c7dd0 0x7fa03c142980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:10.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.931+0000 7fa0423c5640 1 --2- 172.21.15.69:0/985177504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0240039b0 0x7fa03c0cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:10.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.931+0000 7fa0423c5640 1 -- 172.21.15.69:0/985177504 >> 172.21.15.69:0/985177504 conn(0x7fa03c0d1ae0 msgr2=0x7fa03c0b5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:10.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.931+0000 7fa0423c5640 1 -- 172.21.15.69:0/985177504 shutdown_connections 2024-04-25T11:00:10.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:10.931+0000 7fa0423c5640 1 -- 172.21.15.69:0/985177504 wait complete. 2024-04-25T11:00:10.943 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T11:00:10.943 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:00:11.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.049+0000 7f9c88134640 1 Processor -- start 2024-04-25T11:00:11.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.049+0000 7f9c88134640 1 -- start start 2024-04-25T11:00:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.051+0000 7f9c88134640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f9c8006d8e0 0x7f9c800d3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.051+0000 7f9c88134640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c8006dcf0 0x7f9c800d3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.051+0000 7f9c88134640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9c800d04a0 0x7f9c800d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.051+0000 7f9c88134640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f9c800b9520 con 0x7f9c8006d8e0 2024-04-25T11:00:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.051+0000 7f9c88134640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f9c800b4820 con 0x7f9c800d04a0 2024-04-25T11:00:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.051+0000 7f9c88134640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9c800b3c10 con 0x7f9c8006dcf0 2024-04-25T11:00:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.051+0000 7f9c85ed0640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f9c8006d8e0 0x7f9c800d3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.051+0000 7f9c85ed0640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f9c8006d8e0 0x7f9c800d3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36564/0 (socket says 172.21.15.69:36564) 2024-04-25T11:00:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.051+0000 7f9c85ed0640 1 -- 172.21.15.69:0/2402008459 learned_addr learned my addr 172.21.15.69:0/2402008459 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.051+0000 7f9c856cf640 1 --2- 172.21.15.69:0/2402008459 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c8006dcf0 0x7f9c800d3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.051+0000 7f9c866d1640 1 --2- 172.21.15.69:0/2402008459 >> v2:172.21.15.146:3300/0 conn(0x7f9c800d04a0 0x7f9c800d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.051+0000 7f9c85ed0640 1 -- 172.21.15.69:0/2402008459 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c8006dcf0 msgr2=0x7f9c800d3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.051+0000 7f9c85ed0640 1 --2- 172.21.15.69:0/2402008459 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c8006dcf0 0x7f9c800d3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.051+0000 7f9c85ed0640 1 -- 172.21.15.69:0/2402008459 >> v2:172.21.15.146:3300/0 conn(0x7f9c800d04a0 msgr2=0x7f9c800d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:11.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.051+0000 7f9c85ed0640 1 --2- 172.21.15.69:0/2402008459 >> v2:172.21.15.146:3300/0 conn(0x7f9c800d04a0 0x7f9c800d08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:11.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.051+0000 7f9c85ed0640 1 -- 172.21.15.69:0/2402008459 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c800b3ef0 con 0x7f9c8006d8e0 2024-04-25T11:00:11.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.052+0000 7f9c85ed0640 1 --2- 172.21.15.69:0/2402008459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c8006d8e0 0x7f9c800d3a20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9c800b9ae0 tx=0x7f9c7c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e4f2bcd068b20c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:11.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.052+0000 7f9c84ece640 1 -- 172.21.15.69:0/2402008459 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c037070 con 0x7f9c8006d8e0 2024-04-25T11:00:11.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.052+0000 7f9c84ece640 1 -- 172.21.15.69:0/2402008459 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c7c032440 con 0x7f9c8006d8e0 2024-04-25T11:00:11.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.052+0000 7f9c84ece640 1 -- 172.21.15.69:0/2402008459 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c032810 con 0x7f9c8006d8e0 2024-04-25T11:00:11.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.053+0000 7f9c88134640 1 -- 172.21.15.69:0/2402008459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c8006d8e0 msgr2=0x7f9c800d3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:11.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.053+0000 7f9c88134640 1 --2- 172.21.15.69:0/2402008459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c8006d8e0 0x7f9c800d3a20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9c800b9ae0 tx=0x7f9c7c02a140 comp rx=0 tx=0).stop 2024-04-25T11:00:11.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.053+0000 7f9c88134640 1 -- 172.21.15.69:0/2402008459 shutdown_connections 2024-04-25T11:00:11.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.053+0000 7f9c88134640 1 --2- 172.21.15.69:0/2402008459 >> v2:172.21.15.146:3300/0 conn(0x7f9c800d04a0 0x7f9c800d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:11.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.053+0000 7f9c88134640 1 --2- 172.21.15.69:0/2402008459 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c8006dcf0 0x7f9c800d3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:11.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.053+0000 7f9c88134640 1 --2- 172.21.15.69:0/2402008459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c8006d8e0 0x7f9c800d3a20 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:11.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.053+0000 7f9c88134640 1 -- 172.21.15.69:0/2402008459 >> 172.21.15.69:0/2402008459 conn(0x7f9c8008d110 msgr2=0x7f9c8008d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:11.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.053+0000 7f9c88134640 1 -- 172.21.15.69:0/2402008459 shutdown_connections 2024-04-25T11:00:11.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.053+0000 7f9c88134640 1 -- 172.21.15.69:0/2402008459 wait complete. 2024-04-25T11:00:11.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.055+0000 7f9c88134640 1 Processor -- start 2024-04-25T11:00:11.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.055+0000 7f9c88134640 1 -- start start 2024-04-25T11:00:11.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.056+0000 7f9c88134640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c8006d8e0 0x7f9c800a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:11.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.056+0000 7f9c88134640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c8006dcf0 0x7f9c801e8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:11.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.056+0000 7f9c88134640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c800d04a0 0x7f9c801e8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:11.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.056+0000 7f9c88134640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9c800b38e0 con 0x7f9c8006d8e0 2024-04-25T11:00:11.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.056+0000 7f9c88134640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9c800b44b0 con 0x7f9c8006dcf0 2024-04-25T11:00:11.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.056+0000 7f9c88134640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9c800d0df0 con 0x7f9c800d04a0 2024-04-25T11:00:11.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.056+0000 7f9c85ed0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c8006d8e0 0x7f9c800a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:11.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.056+0000 7f9c85ed0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c8006d8e0 0x7f9c800a0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36578/0 (socket says 172.21.15.69:36578) 2024-04-25T11:00:11.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.056+0000 7f9c85ed0640 1 -- 172.21.15.69:0/4188845489 learned_addr learned my addr 172.21.15.69:0/4188845489 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:11.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.056+0000 7f9c85ed0640 1 -- 172.21.15.69:0/4188845489 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c800d04a0 msgr2=0x7f9c801e8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:11.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.056+0000 7f9c85ed0640 1 --2- 172.21.15.69:0/4188845489 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c800d04a0 0x7f9c801e8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:11.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.056+0000 7f9c85ed0640 1 -- 172.21.15.69:0/4188845489 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c8006dcf0 msgr2=0x7f9c801e8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:11.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.056+0000 7f9c85ed0640 1 --2- 172.21.15.69:0/4188845489 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c8006dcf0 0x7f9c801e8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:11.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.056+0000 7f9c85ed0640 1 -- 172.21.15.69:0/4188845489 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8009e760 con 0x7f9c8006d8e0 2024-04-25T11:00:11.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.056+0000 7f9c85ed0640 1 --2- 172.21.15.69:0/4188845489 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c8006d8e0 0x7f9c800a0ed0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c02a180 tx=0x7f9c7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:11.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.057+0000 7f9c767fc640 1 -- 172.21.15.69:0/4188845489 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c041020 con 0x7f9c8006d8e0 2024-04-25T11:00:11.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.057+0000 7f9c767fc640 1 -- 172.21.15.69:0/4188845489 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c7c0306c0 con 0x7f9c8006d8e0 2024-04-25T11:00:11.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.057+0000 7f9c88134640 1 -- 172.21.15.69:0/4188845489 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c801e7ef0 con 0x7f9c8006d8e0 2024-04-25T11:00:11.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.057+0000 7f9c767fc640 1 -- 172.21.15.69:0/4188845489 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c037040 con 0x7f9c8006d8e0 2024-04-25T11:00:11.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.057+0000 7f9c88134640 1 -- 172.21.15.69:0/4188845489 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c801f65f0 con 0x7f9c8006d8e0 2024-04-25T11:00:11.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.058+0000 7f9c88134640 1 -- 172.21.15.69:0/4188845489 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c80050740 con 0x7f9c8006d8e0 2024-04-25T11:00:11.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.058+0000 7f9c767fc640 1 -- 172.21.15.69:0/4188845489 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9c7c044020 con 0x7f9c8006d8e0 2024-04-25T11:00:11.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.059+0000 7f9c767fc640 1 --2- 172.21.15.69:0/4188845489 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f9c640744b0 0x7f9c640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:11.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.059+0000 7f9c767fc640 1 -- 172.21.15.69:0/4188845489 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f9c7c0b5a50 con 0x7f9c8006d8e0 2024-04-25T11:00:11.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.059+0000 7f9c856cf640 1 --2- 172.21.15.69:0/4188845489 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f9c640744b0 0x7f9c640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:11.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.059+0000 7f9c856cf640 1 --2- 172.21.15.69:0/4188845489 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f9c640744b0 0x7f9c640769c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9c800a0e60 tx=0x7f9c70008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:11.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.063+0000 7f9c767fc640 1 -- 172.21.15.69:0/4188845489 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9c7c04b030 con 0x7f9c8006d8e0 2024-04-25T11:00:11.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.235+0000 7f9c88134640 1 -- 172.21.15.69:0/4188845489 --> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9c801f6890 con 0x7f9c640744b0 2024-04-25T11:00:11.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.242+0000 7f9c767fc640 1 -- 172.21.15.69:0/4188845489 <== mgr.4107 v2:172.21.15.69:6832/1107421267 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246618 (secure 0 0 0) 0x7f9c801f6890 con 0x7f9c640744b0 2024-04-25T11:00:11.245 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:00:11.247 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-04-25T11:00:11.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.244+0000 7f9c88134640 1 -- 172.21.15.69:0/4188845489 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f9c640744b0 msgr2=0x7f9c640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:11.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.244+0000 7f9c88134640 1 --2- 172.21.15.69:0/4188845489 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f9c640744b0 0x7f9c640769c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9c800a0e60 tx=0x7f9c70008040 comp rx=0 tx=0).stop 2024-04-25T11:00:11.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.244+0000 7f9c88134640 1 -- 172.21.15.69:0/4188845489 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c8006d8e0 msgr2=0x7f9c800a0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:11.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.244+0000 7f9c88134640 1 --2- 172.21.15.69:0/4188845489 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c8006d8e0 0x7f9c800a0ed0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c02a180 tx=0x7f9c7c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:00:11.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.245+0000 7f9c88134640 1 -- 172.21.15.69:0/4188845489 shutdown_connections 2024-04-25T11:00:11.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.245+0000 7f9c88134640 1 --2- 172.21.15.69:0/4188845489 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f9c640744b0 0x7f9c640769c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:11.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.245+0000 7f9c88134640 1 --2- 172.21.15.69:0/4188845489 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c800d04a0 0x7f9c801e8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:11.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.245+0000 7f9c88134640 1 --2- 172.21.15.69:0/4188845489 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c8006dcf0 0x7f9c801e8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:11.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.245+0000 7f9c88134640 1 --2- 172.21.15.69:0/4188845489 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c8006d8e0 0x7f9c800a0ed0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:11.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.245+0000 7f9c88134640 1 -- 172.21.15.69:0/4188845489 >> 172.21.15.69:0/4188845489 conn(0x7f9c8008d110 msgr2=0x7f9c800d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:11.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.245+0000 7f9c88134640 1 -- 172.21.15.69:0/4188845489 shutdown_connections 2024-04-25T11:00:11.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.245+0000 7f9c88134640 1 -- 172.21.15.69:0/4188845489 wait complete. 2024-04-25T11:00:11.256 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":56,"stamp":"2024-04-25T11:00:10.416036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219768,"kb_used_data":3604,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754952,"statfs":{"total":773094113280,"available":772869070848,"internally_reserved":0,"allocated":3690496,"data_stored":2107867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.181025"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.247019+0000","last_change":"2024-04-25T11:00:06.247019+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.247019+0000","last_undegraded":"2024-04-25T11:00:06.247019+0000","last_fullsized":"2024-04-25T11:00:06.247019+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.248970+0000","last_change":"2024-04-25T11:00:06.248970+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.248970+0000","last_undegraded":"2024-04-25T11:00:06.248970+0000","last_fullsized":"2024-04-25T11:00:06.248970+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.253027+0000","last_change":"2024-04-25T11:00:06.253027+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.253027+0000","last_undegraded":"2024-04-25T11:00:06.253027+0000","last_fullsized":"2024-04-25T11:00:06.253027+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.253669+0000","last_change":"2024-04-25T11:00:06.253669+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.253669+0000","last_undegraded":"2024-04-25T11:00:06.253669+0000","last_fullsized":"2024-04-25T11:00:06.253669+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.252482+0000","last_change":"2024-04-25T11:00:06.252482+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.252482+0000","last_undegraded":"2024-04-25T11:00:06.252482+0000","last_fullsized":"2024-04-25T11:00:06.252482+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.248650+0000","last_change":"2024-04-25T11:00:06.248650+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.248650+0000","last_undegraded":"2024-04-25T11:00:06.248650+0000","last_fullsized":"2024-04-25T11:00:06.248650+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.259535+0000","last_change":"2024-04-25T11:00:06.259535+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.259535+0000","last_undegraded":"2024-04-25T11:00:06.259535+0000","last_fullsized":"2024-04-25T11:00:06.259535+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.263557+0000","last_change":"2024-04-25T11:00:06.263557+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.263557+0000","last_undegraded":"2024-04-25T11:00:06.263557+0000","last_fullsized":"2024-04-25T11:00:06.263557+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.246492+0000","last_change":"2024-04-25T11:00:06.246492+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.246492+0000","last_undegraded":"2024-04-25T11:00:06.246492+0000","last_fullsized":"2024-04-25T11:00:06.246492+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.251364+0000","last_change":"2024-04-25T11:00:06.251364+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.251364+0000","last_undegraded":"2024-04-25T11:00:06.251364+0000","last_fullsized":"2024-04-25T11:00:06.251364+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.260217+0000","last_change":"2024-04-25T11:00:06.260217+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.260217+0000","last_undegraded":"2024-04-25T11:00:06.260217+0000","last_fullsized":"2024-04-25T11:00:06.260217+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.252165+0000","last_change":"2024-04-25T11:00:06.252165+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.252165+0000","last_undegraded":"2024-04-25T11:00:06.252165+0000","last_fullsized":"2024-04-25T11:00:06.252165+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.251519+0000","last_change":"2024-04-25T11:00:06.251519+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.251519+0000","last_undegraded":"2024-04-25T11:00:06.251519+0000","last_fullsized":"2024-04-25T11:00:06.251519+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.248054+0000","last_change":"2024-04-25T11:00:06.248054+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.248054+0000","last_undegraded":"2024-04-25T11:00:06.248054+0000","last_fullsized":"2024-04-25T11:00:06.248054+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.252810+0000","last_change":"2024-04-25T11:00:06.252810+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.252810+0000","last_undegraded":"2024-04-25T11:00:06.252810+0000","last_fullsized":"2024-04-25T11:00:06.252810+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.257769+0000","last_change":"2024-04-25T11:00:06.257769+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.257769+0000","last_undegraded":"2024-04-25T11:00:06.257769+0000","last_fullsized":"2024-04-25T11:00:06.257769+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.261835+0000","last_change":"2024-04-25T11:00:06.261835+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.261835+0000","last_undegraded":"2024-04-25T11:00:06.261835+0000","last_fullsized":"2024-04-25T11:00:06.261835+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.247792+0000","last_change":"2024-04-25T11:00:06.247792+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.247792+0000","last_undegraded":"2024-04-25T11:00:06.247792+0000","last_fullsized":"2024-04-25T11:00:06.247792+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.247704+0000","last_change":"2024-04-25T11:00:06.247704+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.247704+0000","last_undegraded":"2024-04-25T11:00:06.247704+0000","last_fullsized":"2024-04-25T11:00:06.247704+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.251408+0000","last_change":"2024-04-25T11:00:06.251408+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.251408+0000","last_undegraded":"2024-04-25T11:00:06.251408+0000","last_fullsized":"2024-04-25T11:00:06.251408+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.247701+0000","last_change":"2024-04-25T11:00:06.247701+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.247701+0000","last_undegraded":"2024-04-25T11:00:06.247701+0000","last_fullsized":"2024-04-25T11:00:06.247701+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.244357+0000","last_change":"2024-04-25T11:00:06.244357+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.244357+0000","last_undegraded":"2024-04-25T11:00:06.244357+0000","last_fullsized":"2024-04-25T11:00:06.244357+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.249829+0000","last_change":"2024-04-25T11:00:06.249829+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.249829+0000","last_undegraded":"2024-04-25T11:00:06.249829+0000","last_fullsized":"2024-04-25T11:00:06.249829+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.251978+0000","last_change":"2024-04-25T11:00:06.251978+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.251978+0000","last_undegraded":"2024-04-25T11:00:06.251978+0000","last_fullsized":"2024-04-25T11:00:06.251978+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.249905+0000","last_change":"2024-04-25T11:00:06.249905+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.249905+0000","last_undegraded":"2024-04-25T11:00:06.249905+0000","last_fullsized":"2024-04-25T11:00:06.249905+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.247067+0000","last_change":"2024-04-25T11:00:06.247067+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.247067+0000","last_undegraded":"2024-04-25T11:00:06.247067+0000","last_fullsized":"2024-04-25T11:00:06.247067+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.250240+0000","last_change":"2024-04-25T11:00:06.250240+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.250240+0000","last_undegraded":"2024-04-25T11:00:06.250240+0000","last_fullsized":"2024-04-25T11:00:06.250240+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.251501+0000","last_change":"2024-04-25T11:00:06.251501+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.251501+0000","last_undegraded":"2024-04-25T11:00:06.251501+0000","last_fullsized":"2024-04-25T11:00:06.251501+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.246143+0000","last_change":"2024-04-25T11:00:06.246143+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.246143+0000","last_undegraded":"2024-04-25T11:00:06.246143+0000","last_fullsized":"2024-04-25T11:00:06.246143+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.259514+0000","last_change":"2024-04-25T11:00:06.259514+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.259514+0000","last_undegraded":"2024-04-25T11:00:06.259514+0000","last_fullsized":"2024-04-25T11:00:06.259514+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.244943+0000","last_change":"2024-04-25T11:00:06.244943+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.244943+0000","last_undegraded":"2024-04-25T11:00:06.244943+0000","last_fullsized":"2024-04-25T11:00:06.244943+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.263045+0000","last_change":"2024-04-25T11:00:06.263045+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.263045+0000","last_undegraded":"2024-04-25T11:00:06.263045+0000","last_fullsized":"2024-04-25T11:00:06.263045+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.248830+0000","last_change":"2024-04-25T11:00:06.248830+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.248830+0000","last_undegraded":"2024-04-25T11:00:06.248830+0000","last_fullsized":"2024-04-25T11:00:06.248830+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.262835+0000","last_change":"2024-04-25T11:00:06.257849+0000","last_active":"2024-04-25T11:00:06.262835+0000","last_peered":"2024-04-25T11:00:06.262835+0000","last_clean":"2024-04-25T11:00:06.262835+0000","last_became_active":"2024-04-25T11:00:06.257452+0000","last_became_peered":"2024-04-25T11:00:06.257452+0000","last_unstale":"2024-04-25T11:00:06.262835+0000","last_undegraded":"2024-04-25T11:00:06.262835+0000","last_fullsized":"2024-04-25T11:00:06.262835+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:44.176669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.246458+0000","last_change":"2024-04-25T11:00:06.246458+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.246458+0000","last_undegraded":"2024-04-25T11:00:06.246458+0000","last_fullsized":"2024-04-25T11:00:06.246458+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.266419+0000","last_change":"2024-04-25T11:00:06.265699+0000","last_active":"2024-04-25T11:00:06.266419+0000","last_peered":"2024-04-25T11:00:06.266419+0000","last_clean":"2024-04-25T11:00:06.266419+0000","last_became_active":"2024-04-25T11:00:06.265112+0000","last_became_peered":"2024-04-25T11:00:06.265112+0000","last_unstale":"2024-04-25T11:00:06.266419+0000","last_undegraded":"2024-04-25T11:00:06.266419+0000","last_fullsized":"2024-04-25T11:00:06.266419+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:03.333540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.246789+0000","last_change":"2024-04-25T11:00:06.246789+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.246789+0000","last_undegraded":"2024-04-25T11:00:06.246789+0000","last_fullsized":"2024-04-25T11:00:06.246789+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.267960+0000","last_change":"2024-04-25T11:00:06.262272+0000","last_active":"2024-04-25T11:00:06.267960+0000","last_peered":"2024-04-25T11:00:06.267960+0000","last_clean":"2024-04-25T11:00:06.267960+0000","last_became_active":"2024-04-25T11:00:06.261982+0000","last_became_peered":"2024-04-25T11:00:06.261982+0000","last_unstale":"2024-04-25T11:00:06.267960+0000","last_undegraded":"2024-04-25T11:00:06.267960+0000","last_fullsized":"2024-04-25T11:00:06.267960+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:08.488342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.249670+0000","last_change":"2024-04-25T11:00:06.249670+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.249670+0000","last_undegraded":"2024-04-25T11:00:06.249670+0000","last_fullsized":"2024-04-25T11:00:06.249670+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.260566+0000","last_change":"2024-04-25T11:00:06.257693+0000","last_active":"2024-04-25T11:00:06.260566+0000","last_peered":"2024-04-25T11:00:06.260566+0000","last_clean":"2024-04-25T11:00:06.260566+0000","last_became_active":"2024-04-25T11:00:06.257402+0000","last_became_peered":"2024-04-25T11:00:06.257402+0000","last_unstale":"2024-04-25T11:00:06.260566+0000","last_undegraded":"2024-04-25T11:00:06.260566+0000","last_fullsized":"2024-04-25T11:00:06.260566+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:19.772176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.257968+0000","last_change":"2024-04-25T11:00:06.257968+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.257968+0000","last_undegraded":"2024-04-25T11:00:06.257968+0000","last_fullsized":"2024-04-25T11:00:06.257968+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.263511+0000","last_change":"2024-04-25T11:00:06.260758+0000","last_active":"2024-04-25T11:00:06.263511+0000","last_peered":"2024-04-25T11:00:06.263511+0000","last_clean":"2024-04-25T11:00:06.263511+0000","last_became_active":"2024-04-25T11:00:06.260455+0000","last_became_peered":"2024-04-25T11:00:06.260455+0000","last_unstale":"2024-04-25T11:00:06.263511+0000","last_undegraded":"2024-04-25T11:00:06.263511+0000","last_fullsized":"2024-04-25T11:00:06.263511+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:19.772176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.252810+0000","last_change":"2024-04-25T11:00:06.252810+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.252810+0000","last_undegraded":"2024-04-25T11:00:06.252810+0000","last_fullsized":"2024-04-25T11:00:06.252810+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.257660+0000","last_change":"2024-04-25T11:00:06.255855+0000","last_active":"2024-04-25T11:00:06.257660+0000","last_peered":"2024-04-25T11:00:06.257660+0000","last_clean":"2024-04-25T11:00:06.257660+0000","last_became_active":"2024-04-25T11:00:06.255509+0000","last_became_peered":"2024-04-25T11:00:06.255509+0000","last_unstale":"2024-04-25T11:00:06.257660+0000","last_undegraded":"2024-04-25T11:00:06.257660+0000","last_fullsized":"2024-04-25T11:00:06.257660+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:24.915258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.244756+0000","last_change":"2024-04-25T11:00:06.244756+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.244756+0000","last_undegraded":"2024-04-25T11:00:06.244756+0000","last_fullsized":"2024-04-25T11:00:06.244756+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.267818+0000","last_change":"2024-04-25T11:00:06.259273+0000","last_active":"2024-04-25T11:00:06.267818+0000","last_peered":"2024-04-25T11:00:06.267818+0000","last_clean":"2024-04-25T11:00:06.267818+0000","last_became_active":"2024-04-25T11:00:06.258923+0000","last_became_peered":"2024-04-25T11:00:06.258923+0000","last_unstale":"2024-04-25T11:00:06.267818+0000","last_undegraded":"2024-04-25T11:00:06.267818+0000","last_fullsized":"2024-04-25T11:00:06.267818+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:11.002336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.250649+0000","last_change":"2024-04-25T11:00:06.250649+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.250649+0000","last_undegraded":"2024-04-25T11:00:06.250649+0000","last_fullsized":"2024-04-25T11:00:06.250649+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.263159+0000","last_change":"2024-04-25T11:00:06.259650+0000","last_active":"2024-04-25T11:00:06.263159+0000","last_peered":"2024-04-25T11:00:06.263159+0000","last_clean":"2024-04-25T11:00:06.263159+0000","last_became_active":"2024-04-25T11:00:06.259152+0000","last_became_peered":"2024-04-25T11:00:06.259152+0000","last_unstale":"2024-04-25T11:00:06.263159+0000","last_undegraded":"2024-04-25T11:00:06.263159+0000","last_fullsized":"2024-04-25T11:00:06.263159+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:00.807719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.251692+0000","last_change":"2024-04-25T11:00:06.251692+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.251692+0000","last_undegraded":"2024-04-25T11:00:06.251692+0000","last_fullsized":"2024-04-25T11:00:06.251692+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.270373+0000","last_change":"2024-04-25T11:00:06.266165+0000","last_active":"2024-04-25T11:00:06.270373+0000","last_peered":"2024-04-25T11:00:06.270373+0000","last_clean":"2024-04-25T11:00:06.270373+0000","last_became_active":"2024-04-25T11:00:06.265773+0000","last_became_peered":"2024-04-25T11:00:06.265773+0000","last_unstale":"2024-04-25T11:00:06.270373+0000","last_undegraded":"2024-04-25T11:00:06.270373+0000","last_fullsized":"2024-04-25T11:00:06.270373+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:55.930698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.248049+0000","last_change":"2024-04-25T11:00:06.248049+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.248049+0000","last_undegraded":"2024-04-25T11:00:06.248049+0000","last_fullsized":"2024-04-25T11:00:06.248049+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.261209+0000","last_change":"2024-04-25T11:00:06.259862+0000","last_active":"2024-04-25T11:00:06.261209+0000","last_peered":"2024-04-25T11:00:06.261209+0000","last_clean":"2024-04-25T11:00:06.261209+0000","last_became_active":"2024-04-25T11:00:06.259570+0000","last_became_peered":"2024-04-25T11:00:06.259570+0000","last_unstale":"2024-04-25T11:00:06.261209+0000","last_undegraded":"2024-04-25T11:00:06.261209+0000","last_fullsized":"2024-04-25T11:00:06.261209+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:37.048525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.247593+0000","last_change":"2024-04-25T11:00:06.247593+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.247593+0000","last_undegraded":"2024-04-25T11:00:06.247593+0000","last_fullsized":"2024-04-25T11:00:06.247593+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.266946+0000","last_change":"2024-04-25T11:00:06.266271+0000","last_active":"2024-04-25T11:00:06.266946+0000","last_peered":"2024-04-25T11:00:06.266946+0000","last_clean":"2024-04-25T11:00:06.266946+0000","last_became_active":"2024-04-25T11:00:06.265815+0000","last_became_peered":"2024-04-25T11:00:06.265815+0000","last_unstale":"2024-04-25T11:00:06.266946+0000","last_undegraded":"2024-04-25T11:00:06.266946+0000","last_fullsized":"2024-04-25T11:00:06.266946+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:45.067082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.253538+0000","last_change":"2024-04-25T11:00:06.253538+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.253538+0000","last_undegraded":"2024-04-25T11:00:06.253538+0000","last_fullsized":"2024-04-25T11:00:06.253538+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.270434+0000","last_change":"2024-04-25T11:00:06.266846+0000","last_active":"2024-04-25T11:00:06.270434+0000","last_peered":"2024-04-25T11:00:06.270434+0000","last_clean":"2024-04-25T11:00:06.270434+0000","last_became_active":"2024-04-25T11:00:06.266553+0000","last_became_peered":"2024-04-25T11:00:06.266553+0000","last_unstale":"2024-04-25T11:00:06.270434+0000","last_undegraded":"2024-04-25T11:00:06.270434+0000","last_fullsized":"2024-04-25T11:00:06.270434+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:02.917078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.250429+0000","last_change":"2024-04-25T11:00:06.250429+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.250429+0000","last_undegraded":"2024-04-25T11:00:06.250429+0000","last_fullsized":"2024-04-25T11:00:06.250429+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.263245+0000","last_change":"2024-04-25T11:00:06.258144+0000","last_active":"2024-04-25T11:00:06.263245+0000","last_peered":"2024-04-25T11:00:06.263245+0000","last_clean":"2024-04-25T11:00:06.263245+0000","last_became_active":"2024-04-25T11:00:06.257775+0000","last_became_peered":"2024-04-25T11:00:06.257775+0000","last_unstale":"2024-04-25T11:00:06.263245+0000","last_undegraded":"2024-04-25T11:00:06.263245+0000","last_fullsized":"2024-04-25T11:00:06.263245+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:06.103278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.247912+0000","last_change":"2024-04-25T11:00:06.247912+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.247912+0000","last_undegraded":"2024-04-25T11:00:06.247912+0000","last_fullsized":"2024-04-25T11:00:06.247912+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.267664+0000","last_change":"2024-04-25T11:00:06.260717+0000","last_active":"2024-04-25T11:00:06.267664+0000","last_peered":"2024-04-25T11:00:06.267664+0000","last_clean":"2024-04-25T11:00:06.267664+0000","last_became_active":"2024-04-25T11:00:06.260433+0000","last_became_peered":"2024-04-25T11:00:06.260433+0000","last_unstale":"2024-04-25T11:00:06.267664+0000","last_undegraded":"2024-04-25T11:00:06.267664+0000","last_fullsized":"2024-04-25T11:00:06.267664+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:11.002336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.261956+0000","last_change":"2024-04-25T11:00:06.261956+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.261956+0000","last_undegraded":"2024-04-25T11:00:06.261956+0000","last_fullsized":"2024-04-25T11:00:06.261956+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.270336+0000","last_change":"2024-04-25T11:00:06.265403+0000","last_active":"2024-04-25T11:00:06.270336+0000","last_peered":"2024-04-25T11:00:06.270336+0000","last_clean":"2024-04-25T11:00:06.270336+0000","last_became_active":"2024-04-25T11:00:06.265062+0000","last_became_peered":"2024-04-25T11:00:06.265062+0000","last_unstale":"2024-04-25T11:00:06.270336+0000","last_undegraded":"2024-04-25T11:00:06.270336+0000","last_fullsized":"2024-04-25T11:00:06.270336+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:37.048525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.244356+0000","last_change":"2024-04-25T11:00:06.244356+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.244356+0000","last_undegraded":"2024-04-25T11:00:06.244356+0000","last_fullsized":"2024-04-25T11:00:06.244356+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.270426+0000","last_change":"2024-04-25T11:00:06.266837+0000","last_active":"2024-04-25T11:00:06.270426+0000","last_peered":"2024-04-25T11:00:06.270426+0000","last_clean":"2024-04-25T11:00:06.270426+0000","last_became_active":"2024-04-25T11:00:06.266003+0000","last_became_peered":"2024-04-25T11:00:06.266003+0000","last_unstale":"2024-04-25T11:00:06.270426+0000","last_undegraded":"2024-04-25T11:00:06.270426+0000","last_fullsized":"2024-04-25T11:00:06.270426+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:42.908127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.245080+0000","last_change":"2024-04-25T11:00:06.245080+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.245080+0000","last_undegraded":"2024-04-25T11:00:06.245080+0000","last_fullsized":"2024-04-25T11:00:06.245080+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.267711+0000","last_change":"2024-04-25T11:00:06.262237+0000","last_active":"2024-04-25T11:00:06.267711+0000","last_peered":"2024-04-25T11:00:06.267711+0000","last_clean":"2024-04-25T11:00:06.267711+0000","last_became_active":"2024-04-25T11:00:06.261872+0000","last_became_peered":"2024-04-25T11:00:06.261872+0000","last_unstale":"2024-04-25T11:00:06.267711+0000","last_undegraded":"2024-04-25T11:00:06.267711+0000","last_fullsized":"2024-04-25T11:00:06.267711+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:08.488342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.254401+0000","last_change":"2024-04-25T11:00:06.254401+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.254401+0000","last_undegraded":"2024-04-25T11:00:06.254401+0000","last_fullsized":"2024-04-25T11:00:06.254401+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.259876+0000","last_change":"2024-04-25T11:00:06.257930+0000","last_active":"2024-04-25T11:00:06.259876+0000","last_peered":"2024-04-25T11:00:06.259876+0000","last_clean":"2024-04-25T11:00:06.259876+0000","last_became_active":"2024-04-25T11:00:06.257326+0000","last_became_peered":"2024-04-25T11:00:06.257326+0000","last_unstale":"2024-04-25T11:00:06.259876+0000","last_undegraded":"2024-04-25T11:00:06.259876+0000","last_fullsized":"2024-04-25T11:00:06.259876+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:38.952429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.251501+0000","last_change":"2024-04-25T11:00:06.251501+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.251501+0000","last_undegraded":"2024-04-25T11:00:06.251501+0000","last_fullsized":"2024-04-25T11:00:06.251501+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.259882+0000","last_change":"2024-04-25T11:00:06.256448+0000","last_active":"2024-04-25T11:00:06.259882+0000","last_peered":"2024-04-25T11:00:06.259882+0000","last_clean":"2024-04-25T11:00:06.259882+0000","last_became_active":"2024-04-25T11:00:06.256095+0000","last_became_peered":"2024-04-25T11:00:06.256095+0000","last_unstale":"2024-04-25T11:00:06.259882+0000","last_undegraded":"2024-04-25T11:00:06.259882+0000","last_fullsized":"2024-04-25T11:00:06.259882+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:04.049748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.250248+0000","last_change":"2024-04-25T11:00:06.250248+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.250248+0000","last_undegraded":"2024-04-25T11:00:06.250248+0000","last_fullsized":"2024-04-25T11:00:06.250248+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.263803+0000","last_change":"2024-04-25T11:00:06.261291+0000","last_active":"2024-04-25T11:00:06.263803+0000","last_peered":"2024-04-25T11:00:06.263803+0000","last_clean":"2024-04-25T11:00:06.263803+0000","last_became_active":"2024-04-25T11:00:06.260969+0000","last_became_peered":"2024-04-25T11:00:06.260969+0000","last_unstale":"2024-04-25T11:00:06.263803+0000","last_undegraded":"2024-04-25T11:00:06.263803+0000","last_fullsized":"2024-04-25T11:00:06.263803+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:31.946740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.247016+0000","last_change":"2024-04-25T11:00:06.247016+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.247016+0000","last_undegraded":"2024-04-25T11:00:06.247016+0000","last_fullsized":"2024-04-25T11:00:06.247016+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.260878+0000","last_change":"2024-04-25T11:00:06.259265+0000","last_active":"2024-04-25T11:00:06.260878+0000","last_peered":"2024-04-25T11:00:06.260878+0000","last_clean":"2024-04-25T11:00:06.260878+0000","last_became_active":"2024-04-25T11:00:06.258949+0000","last_became_peered":"2024-04-25T11:00:06.258949+0000","last_unstale":"2024-04-25T11:00:06.260878+0000","last_undegraded":"2024-04-25T11:00:06.260878+0000","last_fullsized":"2024-04-25T11:00:06.260878+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:08.488342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.249286+0000","last_change":"2024-04-25T11:00:06.249286+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.249286+0000","last_undegraded":"2024-04-25T11:00:06.249286+0000","last_fullsized":"2024-04-25T11:00:06.249286+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.257314+0000","last_change":"2024-04-25T11:00:06.255869+0000","last_active":"2024-04-25T11:00:06.257314+0000","last_peered":"2024-04-25T11:00:06.257314+0000","last_clean":"2024-04-25T11:00:06.257314+0000","last_became_active":"2024-04-25T11:00:06.255468+0000","last_became_peered":"2024-04-25T11:00:06.255468+0000","last_unstale":"2024-04-25T11:00:06.257314+0000","last_undegraded":"2024-04-25T11:00:06.257314+0000","last_fullsized":"2024-04-25T11:00:06.257314+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:40.478099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.246600+0000","last_change":"2024-04-25T11:00:06.246600+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.246600+0000","last_undegraded":"2024-04-25T11:00:06.246600+0000","last_fullsized":"2024-04-25T11:00:06.246600+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.263097+0000","last_change":"2024-04-25T11:00:06.259165+0000","last_active":"2024-04-25T11:00:06.263097+0000","last_peered":"2024-04-25T11:00:06.263097+0000","last_clean":"2024-04-25T11:00:06.263097+0000","last_became_active":"2024-04-25T11:00:06.258814+0000","last_became_peered":"2024-04-25T11:00:06.258814+0000","last_unstale":"2024-04-25T11:00:06.263097+0000","last_undegraded":"2024-04-25T11:00:06.263097+0000","last_fullsized":"2024-04-25T11:00:06.263097+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:46.929021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.261354+0000","last_change":"2024-04-25T11:00:06.261354+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.261354+0000","last_undegraded":"2024-04-25T11:00:06.261354+0000","last_fullsized":"2024-04-25T11:00:06.261354+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.266933+0000","last_change":"2024-04-25T11:00:06.266180+0000","last_active":"2024-04-25T11:00:06.266933+0000","last_peered":"2024-04-25T11:00:06.266933+0000","last_clean":"2024-04-25T11:00:06.266933+0000","last_became_active":"2024-04-25T11:00:06.265801+0000","last_became_peered":"2024-04-25T11:00:06.265801+0000","last_unstale":"2024-04-25T11:00:06.266933+0000","last_undegraded":"2024-04-25T11:00:06.266933+0000","last_fullsized":"2024-04-25T11:00:06.266933+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:27.813350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.245930+0000","last_change":"2024-04-25T11:00:06.245930+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.245930+0000","last_undegraded":"2024-04-25T11:00:06.245930+0000","last_fullsized":"2024-04-25T11:00:06.245930+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.265567+0000","last_change":"2024-04-25T11:00:06.257887+0000","last_active":"2024-04-25T11:00:06.265567+0000","last_peered":"2024-04-25T11:00:06.265567+0000","last_clean":"2024-04-25T11:00:06.265567+0000","last_became_active":"2024-04-25T11:00:06.257563+0000","last_became_peered":"2024-04-25T11:00:06.257563+0000","last_unstale":"2024-04-25T11:00:06.265567+0000","last_undegraded":"2024-04-25T11:00:06.265567+0000","last_fullsized":"2024-04-25T11:00:06.265567+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:44.176669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.253796+0000","last_change":"2024-04-25T11:00:06.253796+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.253796+0000","last_undegraded":"2024-04-25T11:00:06.253796+0000","last_fullsized":"2024-04-25T11:00:06.253796+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.260846+0000","last_change":"2024-04-25T11:00:06.259220+0000","last_active":"2024-04-25T11:00:06.260846+0000","last_peered":"2024-04-25T11:00:06.260846+0000","last_clean":"2024-04-25T11:00:06.260846+0000","last_became_active":"2024-04-25T11:00:06.258895+0000","last_became_peered":"2024-04-25T11:00:06.258895+0000","last_unstale":"2024-04-25T11:00:06.260846+0000","last_undegraded":"2024-04-25T11:00:06.260846+0000","last_fullsized":"2024-04-25T11:00:06.260846+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:11.002336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.248846+0000","last_change":"2024-04-25T11:00:06.248846+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.248846+0000","last_undegraded":"2024-04-25T11:00:06.248846+0000","last_fullsized":"2024-04-25T11:00:06.248846+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.265541+0000","last_change":"2024-04-25T11:00:06.257498+0000","last_active":"2024-04-25T11:00:06.265541+0000","last_peered":"2024-04-25T11:00:06.265541+0000","last_clean":"2024-04-25T11:00:06.265541+0000","last_became_active":"2024-04-25T11:00:06.257083+0000","last_became_peered":"2024-04-25T11:00:06.257083+0000","last_unstale":"2024-04-25T11:00:06.265541+0000","last_undegraded":"2024-04-25T11:00:06.265541+0000","last_fullsized":"2024-04-25T11:00:06.265541+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:30.099257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.245526+0000","last_change":"2024-04-25T11:00:06.245526+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.245526+0000","last_undegraded":"2024-04-25T11:00:06.245526+0000","last_fullsized":"2024-04-25T11:00:06.245526+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.260057+0000","last_change":"2024-04-25T11:00:06.257789+0000","last_active":"2024-04-25T11:00:06.260057+0000","last_peered":"2024-04-25T11:00:06.260057+0000","last_clean":"2024-04-25T11:00:06.260057+0000","last_became_active":"2024-04-25T11:00:06.257450+0000","last_became_peered":"2024-04-25T11:00:06.257450+0000","last_unstale":"2024-04-25T11:00:06.260057+0000","last_undegraded":"2024-04-25T11:00:06.260057+0000","last_fullsized":"2024-04-25T11:00:06.260057+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:44.176669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.248002+0000","last_change":"2024-04-25T11:00:06.248002+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.248002+0000","last_undegraded":"2024-04-25T11:00:06.248002+0000","last_fullsized":"2024-04-25T11:00:06.248002+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.265641+0000","last_change":"2024-04-25T11:00:06.257884+0000","last_active":"2024-04-25T11:00:06.265641+0000","last_peered":"2024-04-25T11:00:06.265641+0000","last_clean":"2024-04-25T11:00:06.265641+0000","last_became_active":"2024-04-25T11:00:06.257586+0000","last_became_peered":"2024-04-25T11:00:06.257586+0000","last_unstale":"2024-04-25T11:00:06.265641+0000","last_undegraded":"2024-04-25T11:00:06.265641+0000","last_fullsized":"2024-04-25T11:00:06.265641+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:40.478099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.248465+0000","last_change":"2024-04-25T11:00:06.248465+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.248465+0000","last_undegraded":"2024-04-25T11:00:06.248465+0000","last_fullsized":"2024-04-25T11:00:06.248465+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.263074+0000","last_change":"2024-04-25T11:00:06.258316+0000","last_active":"2024-04-25T11:00:06.263074+0000","last_peered":"2024-04-25T11:00:06.263074+0000","last_clean":"2024-04-25T11:00:06.263074+0000","last_became_active":"2024-04-25T11:00:06.257838+0000","last_became_peered":"2024-04-25T11:00:06.257838+0000","last_unstale":"2024-04-25T11:00:06.263074+0000","last_undegraded":"2024-04-25T11:00:06.263074+0000","last_fullsized":"2024-04-25T11:00:06.263074+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:19.075396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.247389+0000","last_change":"2024-04-25T11:00:06.247389+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.247389+0000","last_undegraded":"2024-04-25T11:00:06.247389+0000","last_fullsized":"2024-04-25T11:00:06.247389+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.270359+0000","last_change":"2024-04-25T11:00:06.266105+0000","last_active":"2024-04-25T11:00:06.270359+0000","last_peered":"2024-04-25T11:00:06.270359+0000","last_clean":"2024-04-25T11:00:06.270359+0000","last_became_active":"2024-04-25T11:00:06.265786+0000","last_became_peered":"2024-04-25T11:00:06.265786+0000","last_unstale":"2024-04-25T11:00:06.270359+0000","last_undegraded":"2024-04-25T11:00:06.270359+0000","last_fullsized":"2024-04-25T11:00:06.270359+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:03.333540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.257208+0000","last_change":"2024-04-25T11:00:06.257208+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.257208+0000","last_undegraded":"2024-04-25T11:00:06.257208+0000","last_fullsized":"2024-04-25T11:00:06.257208+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.256715+0000","last_change":"2024-04-25T11:00:06.250003+0000","last_active":"2024-04-25T11:00:06.256715+0000","last_peered":"2024-04-25T11:00:06.256715+0000","last_clean":"2024-04-25T11:00:06.256715+0000","last_became_active":"2024-04-25T11:00:06.249632+0000","last_became_peered":"2024-04-25T11:00:06.249632+0000","last_unstale":"2024-04-25T11:00:06.256715+0000","last_undegraded":"2024-04-25T11:00:06.256715+0000","last_fullsized":"2024-04-25T11:00:06.256715+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:30.099257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":125,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.263800+0000","last_change":"2024-04-25T10:59:07.839466+0000","last_active":"2024-04-25T11:00:06.263800+0000","last_peered":"2024-04-25T11:00:06.263800+0000","last_clean":"2024-04-25T11:00:06.263800+0000","last_became_active":"2024-04-25T10:59:07.836269+0000","last_became_peered":"2024-04-25T10:59:07.836269+0000","last_unstale":"2024-04-25T11:00:06.263800+0000","last_undegraded":"2024-04-25T11:00:06.263800+0000","last_fullsized":"2024-04-25T11:00:06.263800+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:06.549663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:06.549663+0000","last_clean_scrub_stamp":"2024-04-25T10:59:06.549663+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:21.575330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":905216,"data_stored":705476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":115108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":115108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":115108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":115108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":115108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":905216,"data_stored":705476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:00:11.261 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:00:11.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.403+0000 7fd5399ce640 1 Processor -- start 2024-04-25T11:00:11.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.403+0000 7fd5399ce640 1 -- start start 2024-04-25T11:00:11.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.404+0000 7fd5399ce640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd5340d0e00 0x7fd5340d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:11.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.404+0000 7fd5399ce640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5340d4a50 0x7fd5340d4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:11.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.404+0000 7fd5399ce640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5340d1850 con 0x7fd53406c5c0 2024-04-25T11:00:11.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.404+0000 7fd5399ce640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fd534160de0 con 0x7fd5340d0e00 2024-04-25T11:00:11.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.404+0000 7fd5399ce640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd5341610c0 con 0x7fd5340d4a50 2024-04-25T11:00:11.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.405+0000 7fd532ffd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd5340d0e00 0x7fd5340d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:11.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.405+0000 7fd5337fe640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5340d4a50 0x7fd5340d4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:11.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.405+0000 7fd5327fc640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fd53406c5c0 0x7fd53406c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.69:51542/0 (socket says 172.21.15.69:51542) 2024-04-25T11:00:11.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.405+0000 7fd5327fc640 1 -- 172.21.15.69:0/1580385977 learned_addr learned my addr 172.21.15.69:0/1580385977 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:11.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.405+0000 7fd5337fe640 1 -- 172.21.15.69:0/1580385977 >> v2:172.21.15.146:3300/0 conn(0x7fd5340d0e00 msgr2=0x7fd5340d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:11.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.405+0000 7fd5337fe640 1 --2- 172.21.15.69:0/1580385977 >> v2:172.21.15.146:3300/0 conn(0x7fd5340d0e00 0x7fd5340d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:11.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.405+0000 7fd5337fe640 1 -- 172.21.15.69:0/1580385977 >> v1:172.21.15.146:6789/0 conn(0x7fd53406c5c0 legacy=0x7fd53406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:11.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.405+0000 7fd5337fe640 1 -- 172.21.15.69:0/1580385977 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5341613a0 con 0x7fd5340d4a50 2024-04-25T11:00:11.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.405+0000 7fd532ffd640 1 --2- 172.21.15.69:0/1580385977 >> v2:172.21.15.146:3300/0 conn(0x7fd5340d0e00 0x7fd5340d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:00:11.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.406+0000 7fd5337fe640 1 --2- 172.21.15.69:0/1580385977 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5340d4a50 0x7fd5340d4e60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd52c0097a0 tx=0x7fd52c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=689ccd187a488622 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:11.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.406+0000 7fd531ffb640 1 -- 172.21.15.69:0/1580385977 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c037070 con 0x7fd5340d4a50 2024-04-25T11:00:11.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.406+0000 7fd531ffb640 1 -- 172.21.15.69:0/1580385977 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd52c032440 con 0x7fd5340d4a50 2024-04-25T11:00:11.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.406+0000 7fd531ffb640 1 -- 172.21.15.69:0/1580385977 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c032810 con 0x7fd5340d4a50 2024-04-25T11:00:11.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.407+0000 7fd5399ce640 1 -- 172.21.15.69:0/1580385977 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5340d4a50 msgr2=0x7fd5340d4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:11.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.407+0000 7fd5399ce640 1 --2- 172.21.15.69:0/1580385977 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5340d4a50 0x7fd5340d4e60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd52c0097a0 tx=0x7fd52c02a180 comp rx=0 tx=0).stop 2024-04-25T11:00:11.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.407+0000 7fd5399ce640 1 -- 172.21.15.69:0/1580385977 shutdown_connections 2024-04-25T11:00:11.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.407+0000 7fd5399ce640 1 --2- 172.21.15.69:0/1580385977 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5340d4a50 0x7fd5340d4e60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:11.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.407+0000 7fd5399ce640 1 --2- 172.21.15.69:0/1580385977 >> v2:172.21.15.146:3300/0 conn(0x7fd5340d0e00 0x7fd5340d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:11.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.407+0000 7fd5399ce640 1 -- 172.21.15.69:0/1580385977 >> 172.21.15.69:0/1580385977 conn(0x7fd5340d5d90 msgr2=0x7fd5340d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:11.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.407+0000 7fd5399ce640 1 -- 172.21.15.69:0/1580385977 shutdown_connections 2024-04-25T11:00:11.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.407+0000 7fd5399ce640 1 -- 172.21.15.69:0/1580385977 wait complete. 2024-04-25T11:00:11.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.409+0000 7fd5399ce640 1 Processor -- start 2024-04-25T11:00:11.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.409+0000 7fd5399ce640 1 -- start start 2024-04-25T11:00:11.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.410+0000 7fd5399ce640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd53406c5c0 0x7fd53415b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:11.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.410+0000 7fd5399ce640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5340d0e00 0x7fd5340cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:11.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.410+0000 7fd5399ce640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5340d4a50 0x7fd5340cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:11.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.410+0000 7fd5399ce640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd534164e00 con 0x7fd53406c5c0 2024-04-25T11:00:11.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.410+0000 7fd5399ce640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd5340b5320 con 0x7fd5340d4a50 2024-04-25T11:00:11.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.410+0000 7fd5399ce640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fd534164f80 con 0x7fd5340d0e00 2024-04-25T11:00:11.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.410+0000 7fd532ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd53406c5c0 0x7fd53415b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:11.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.410+0000 7fd532ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd53406c5c0 0x7fd53415b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36588/0 (socket says 172.21.15.69:36588) 2024-04-25T11:00:11.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.410+0000 7fd532ffd640 1 -- 172.21.15.69:0/164886825 learned_addr learned my addr 172.21.15.69:0/164886825 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:11.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.411+0000 7fd5327fc640 1 --2- 172.21.15.69:0/164886825 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5340d0e00 0x7fd5340cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:11.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.411+0000 7fd5337fe640 1 --2- 172.21.15.69:0/164886825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5340d4a50 0x7fd5340cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:11.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.411+0000 7fd532ffd640 1 -- 172.21.15.69:0/164886825 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5340d0e00 msgr2=0x7fd5340cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:11.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.411+0000 7fd532ffd640 1 --2- 172.21.15.69:0/164886825 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5340d0e00 0x7fd5340cba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:11.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.411+0000 7fd532ffd640 1 -- 172.21.15.69:0/164886825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5340d4a50 msgr2=0x7fd5340cbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:11.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.411+0000 7fd532ffd640 1 --2- 172.21.15.69:0/164886825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5340d4a50 0x7fd5340cbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:11.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.411+0000 7fd532ffd640 1 -- 172.21.15.69:0/164886825 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5340ca1a0 con 0x7fd53406c5c0 2024-04-25T11:00:11.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.411+0000 7fd532ffd640 1 --2- 172.21.15.69:0/164886825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd53406c5c0 0x7fd53415b470 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd524009ec0 tx=0x7fd52400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:11.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.411+0000 7fd51bfff640 1 -- 172.21.15.69:0/164886825 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd524018020 con 0x7fd53406c5c0 2024-04-25T11:00:11.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.411+0000 7fd51bfff640 1 -- 172.21.15.69:0/164886825 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd524014680 con 0x7fd53406c5c0 2024-04-25T11:00:11.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.411+0000 7fd5399ce640 1 -- 172.21.15.69:0/164886825 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5340b6cf0 con 0x7fd53406c5c0 2024-04-25T11:00:11.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.411+0000 7fd51bfff640 1 -- 172.21.15.69:0/164886825 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd524014a50 con 0x7fd53406c5c0 2024-04-25T11:00:11.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.411+0000 7fd5399ce640 1 -- 172.21.15.69:0/164886825 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5340ca480 con 0x7fd53406c5c0 2024-04-25T11:00:11.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.412+0000 7fd5399ce640 1 -- 172.21.15.69:0/164886825 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd534074820 con 0x7fd53406c5c0 2024-04-25T11:00:11.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.413+0000 7fd51bfff640 1 -- 172.21.15.69:0/164886825 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd524005d10 con 0x7fd53406c5c0 2024-04-25T11:00:11.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.413+0000 7fd51bfff640 1 --2- 172.21.15.69:0/164886825 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd5200744b0 0x7fd5200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:11.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.413+0000 7fd51bfff640 1 -- 172.21.15.69:0/164886825 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fd524095f30 con 0x7fd53406c5c0 2024-04-25T11:00:11.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.413+0000 7fd5327fc640 1 --2- 172.21.15.69:0/164886825 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd5200744b0 0x7fd5200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:11.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.414+0000 7fd5327fc640 1 --2- 172.21.15.69:0/164886825 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd5200744b0 0x7fd5200769c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd5340cf920 tx=0x7fd528009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:11.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.418+0000 7fd51bfff640 1 -- 172.21.15.69:0/164886825 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd534074820 con 0x7fd53406c5c0 2024-04-25T11:00:11.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.593+0000 7fd5399ce640 1 -- 172.21.15.69:0/164886825 --> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd534050740 con 0x7fd5200744b0 2024-04-25T11:00:11.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.599+0000 7fd51bfff640 1 -- 172.21.15.69:0/164886825 <== mgr.4107 v2:172.21.15.69:6832/1107421267 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246618 (secure 0 0 0) 0x7fd534050740 con 0x7fd5200744b0 2024-04-25T11:00:11.602 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:00:11.603 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-04-25T11:00:11.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.602+0000 7fd5399ce640 1 -- 172.21.15.69:0/164886825 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd5200744b0 msgr2=0x7fd5200769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:11.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.602+0000 7fd5399ce640 1 --2- 172.21.15.69:0/164886825 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd5200744b0 0x7fd5200769c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd5340cf920 tx=0x7fd528009040 comp rx=0 tx=0).stop 2024-04-25T11:00:11.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.602+0000 7fd5399ce640 1 -- 172.21.15.69:0/164886825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd53406c5c0 msgr2=0x7fd53415b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:11.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.602+0000 7fd5399ce640 1 --2- 172.21.15.69:0/164886825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd53406c5c0 0x7fd53415b470 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd524009ec0 tx=0x7fd52400c490 comp rx=0 tx=0).stop 2024-04-25T11:00:11.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.602+0000 7fd5399ce640 1 -- 172.21.15.69:0/164886825 shutdown_connections 2024-04-25T11:00:11.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.602+0000 7fd5399ce640 1 --2- 172.21.15.69:0/164886825 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fd5200744b0 0x7fd5200769c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:11.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.602+0000 7fd5399ce640 1 --2- 172.21.15.69:0/164886825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5340d4a50 0x7fd5340cbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:11.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.602+0000 7fd5399ce640 1 --2- 172.21.15.69:0/164886825 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5340d0e00 0x7fd5340cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:11.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.602+0000 7fd5399ce640 1 --2- 172.21.15.69:0/164886825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd53406c5c0 0x7fd53415b470 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:11.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.602+0000 7fd5399ce640 1 -- 172.21.15.69:0/164886825 >> 172.21.15.69:0/164886825 conn(0x7fd5340d5d90 msgr2=0x7fd534163890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:11.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.602+0000 7fd5399ce640 1 -- 172.21.15.69:0/164886825 shutdown_connections 2024-04-25T11:00:11.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.602+0000 7fd5399ce640 1 -- 172.21.15.69:0/164886825 wait complete. 2024-04-25T11:00:11.614 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":56,"stamp":"2024-04-25T11:00:10.416036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219768,"kb_used_data":3604,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754952,"statfs":{"total":773094113280,"available":772869070848,"internally_reserved":0,"allocated":3690496,"data_stored":2107867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.181025"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.247019+0000","last_change":"2024-04-25T11:00:06.247019+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.247019+0000","last_undegraded":"2024-04-25T11:00:06.247019+0000","last_fullsized":"2024-04-25T11:00:06.247019+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.248970+0000","last_change":"2024-04-25T11:00:06.248970+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.248970+0000","last_undegraded":"2024-04-25T11:00:06.248970+0000","last_fullsized":"2024-04-25T11:00:06.248970+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.253027+0000","last_change":"2024-04-25T11:00:06.253027+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.253027+0000","last_undegraded":"2024-04-25T11:00:06.253027+0000","last_fullsized":"2024-04-25T11:00:06.253027+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.253669+0000","last_change":"2024-04-25T11:00:06.253669+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.253669+0000","last_undegraded":"2024-04-25T11:00:06.253669+0000","last_fullsized":"2024-04-25T11:00:06.253669+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.252482+0000","last_change":"2024-04-25T11:00:06.252482+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.252482+0000","last_undegraded":"2024-04-25T11:00:06.252482+0000","last_fullsized":"2024-04-25T11:00:06.252482+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.248650+0000","last_change":"2024-04-25T11:00:06.248650+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.248650+0000","last_undegraded":"2024-04-25T11:00:06.248650+0000","last_fullsized":"2024-04-25T11:00:06.248650+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.259535+0000","last_change":"2024-04-25T11:00:06.259535+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.259535+0000","last_undegraded":"2024-04-25T11:00:06.259535+0000","last_fullsized":"2024-04-25T11:00:06.259535+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.263557+0000","last_change":"2024-04-25T11:00:06.263557+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.263557+0000","last_undegraded":"2024-04-25T11:00:06.263557+0000","last_fullsized":"2024-04-25T11:00:06.263557+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.246492+0000","last_change":"2024-04-25T11:00:06.246492+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.246492+0000","last_undegraded":"2024-04-25T11:00:06.246492+0000","last_fullsized":"2024-04-25T11:00:06.246492+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.251364+0000","last_change":"2024-04-25T11:00:06.251364+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.251364+0000","last_undegraded":"2024-04-25T11:00:06.251364+0000","last_fullsized":"2024-04-25T11:00:06.251364+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.260217+0000","last_change":"2024-04-25T11:00:06.260217+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.260217+0000","last_undegraded":"2024-04-25T11:00:06.260217+0000","last_fullsized":"2024-04-25T11:00:06.260217+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.252165+0000","last_change":"2024-04-25T11:00:06.252165+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.252165+0000","last_undegraded":"2024-04-25T11:00:06.252165+0000","last_fullsized":"2024-04-25T11:00:06.252165+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.251519+0000","last_change":"2024-04-25T11:00:06.251519+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.251519+0000","last_undegraded":"2024-04-25T11:00:06.251519+0000","last_fullsized":"2024-04-25T11:00:06.251519+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.248054+0000","last_change":"2024-04-25T11:00:06.248054+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.248054+0000","last_undegraded":"2024-04-25T11:00:06.248054+0000","last_fullsized":"2024-04-25T11:00:06.248054+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.252810+0000","last_change":"2024-04-25T11:00:06.252810+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.252810+0000","last_undegraded":"2024-04-25T11:00:06.252810+0000","last_fullsized":"2024-04-25T11:00:06.252810+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.257769+0000","last_change":"2024-04-25T11:00:06.257769+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.257769+0000","last_undegraded":"2024-04-25T11:00:06.257769+0000","last_fullsized":"2024-04-25T11:00:06.257769+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.261835+0000","last_change":"2024-04-25T11:00:06.261835+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.261835+0000","last_undegraded":"2024-04-25T11:00:06.261835+0000","last_fullsized":"2024-04-25T11:00:06.261835+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.247792+0000","last_change":"2024-04-25T11:00:06.247792+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.247792+0000","last_undegraded":"2024-04-25T11:00:06.247792+0000","last_fullsized":"2024-04-25T11:00:06.247792+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.247704+0000","last_change":"2024-04-25T11:00:06.247704+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.247704+0000","last_undegraded":"2024-04-25T11:00:06.247704+0000","last_fullsized":"2024-04-25T11:00:06.247704+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.251408+0000","last_change":"2024-04-25T11:00:06.251408+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.251408+0000","last_undegraded":"2024-04-25T11:00:06.251408+0000","last_fullsized":"2024-04-25T11:00:06.251408+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.247701+0000","last_change":"2024-04-25T11:00:06.247701+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.247701+0000","last_undegraded":"2024-04-25T11:00:06.247701+0000","last_fullsized":"2024-04-25T11:00:06.247701+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.244357+0000","last_change":"2024-04-25T11:00:06.244357+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.244357+0000","last_undegraded":"2024-04-25T11:00:06.244357+0000","last_fullsized":"2024-04-25T11:00:06.244357+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.249829+0000","last_change":"2024-04-25T11:00:06.249829+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.249829+0000","last_undegraded":"2024-04-25T11:00:06.249829+0000","last_fullsized":"2024-04-25T11:00:06.249829+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.251978+0000","last_change":"2024-04-25T11:00:06.251978+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.251978+0000","last_undegraded":"2024-04-25T11:00:06.251978+0000","last_fullsized":"2024-04-25T11:00:06.251978+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.249905+0000","last_change":"2024-04-25T11:00:06.249905+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.249905+0000","last_undegraded":"2024-04-25T11:00:06.249905+0000","last_fullsized":"2024-04-25T11:00:06.249905+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.247067+0000","last_change":"2024-04-25T11:00:06.247067+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.247067+0000","last_undegraded":"2024-04-25T11:00:06.247067+0000","last_fullsized":"2024-04-25T11:00:06.247067+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.250240+0000","last_change":"2024-04-25T11:00:06.250240+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.250240+0000","last_undegraded":"2024-04-25T11:00:06.250240+0000","last_fullsized":"2024-04-25T11:00:06.250240+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.251501+0000","last_change":"2024-04-25T11:00:06.251501+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.251501+0000","last_undegraded":"2024-04-25T11:00:06.251501+0000","last_fullsized":"2024-04-25T11:00:06.251501+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.246143+0000","last_change":"2024-04-25T11:00:06.246143+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.246143+0000","last_undegraded":"2024-04-25T11:00:06.246143+0000","last_fullsized":"2024-04-25T11:00:06.246143+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.259514+0000","last_change":"2024-04-25T11:00:06.259514+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.259514+0000","last_undegraded":"2024-04-25T11:00:06.259514+0000","last_fullsized":"2024-04-25T11:00:06.259514+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.244943+0000","last_change":"2024-04-25T11:00:06.244943+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.244943+0000","last_undegraded":"2024-04-25T11:00:06.244943+0000","last_fullsized":"2024-04-25T11:00:06.244943+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.263045+0000","last_change":"2024-04-25T11:00:06.263045+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.263045+0000","last_undegraded":"2024-04-25T11:00:06.263045+0000","last_fullsized":"2024-04-25T11:00:06.263045+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.248830+0000","last_change":"2024-04-25T11:00:06.248830+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.248830+0000","last_undegraded":"2024-04-25T11:00:06.248830+0000","last_fullsized":"2024-04-25T11:00:06.248830+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.262835+0000","last_change":"2024-04-25T11:00:06.257849+0000","last_active":"2024-04-25T11:00:06.262835+0000","last_peered":"2024-04-25T11:00:06.262835+0000","last_clean":"2024-04-25T11:00:06.262835+0000","last_became_active":"2024-04-25T11:00:06.257452+0000","last_became_peered":"2024-04-25T11:00:06.257452+0000","last_unstale":"2024-04-25T11:00:06.262835+0000","last_undegraded":"2024-04-25T11:00:06.262835+0000","last_fullsized":"2024-04-25T11:00:06.262835+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:44.176669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.246458+0000","last_change":"2024-04-25T11:00:06.246458+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.246458+0000","last_undegraded":"2024-04-25T11:00:06.246458+0000","last_fullsized":"2024-04-25T11:00:06.246458+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.266419+0000","last_change":"2024-04-25T11:00:06.265699+0000","last_active":"2024-04-25T11:00:06.266419+0000","last_peered":"2024-04-25T11:00:06.266419+0000","last_clean":"2024-04-25T11:00:06.266419+0000","last_became_active":"2024-04-25T11:00:06.265112+0000","last_became_peered":"2024-04-25T11:00:06.265112+0000","last_unstale":"2024-04-25T11:00:06.266419+0000","last_undegraded":"2024-04-25T11:00:06.266419+0000","last_fullsized":"2024-04-25T11:00:06.266419+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:03.333540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.246789+0000","last_change":"2024-04-25T11:00:06.246789+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.246789+0000","last_undegraded":"2024-04-25T11:00:06.246789+0000","last_fullsized":"2024-04-25T11:00:06.246789+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.267960+0000","last_change":"2024-04-25T11:00:06.262272+0000","last_active":"2024-04-25T11:00:06.267960+0000","last_peered":"2024-04-25T11:00:06.267960+0000","last_clean":"2024-04-25T11:00:06.267960+0000","last_became_active":"2024-04-25T11:00:06.261982+0000","last_became_peered":"2024-04-25T11:00:06.261982+0000","last_unstale":"2024-04-25T11:00:06.267960+0000","last_undegraded":"2024-04-25T11:00:06.267960+0000","last_fullsized":"2024-04-25T11:00:06.267960+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:08.488342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.249670+0000","last_change":"2024-04-25T11:00:06.249670+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.249670+0000","last_undegraded":"2024-04-25T11:00:06.249670+0000","last_fullsized":"2024-04-25T11:00:06.249670+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.260566+0000","last_change":"2024-04-25T11:00:06.257693+0000","last_active":"2024-04-25T11:00:06.260566+0000","last_peered":"2024-04-25T11:00:06.260566+0000","last_clean":"2024-04-25T11:00:06.260566+0000","last_became_active":"2024-04-25T11:00:06.257402+0000","last_became_peered":"2024-04-25T11:00:06.257402+0000","last_unstale":"2024-04-25T11:00:06.260566+0000","last_undegraded":"2024-04-25T11:00:06.260566+0000","last_fullsized":"2024-04-25T11:00:06.260566+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:19.772176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.257968+0000","last_change":"2024-04-25T11:00:06.257968+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.257968+0000","last_undegraded":"2024-04-25T11:00:06.257968+0000","last_fullsized":"2024-04-25T11:00:06.257968+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.263511+0000","last_change":"2024-04-25T11:00:06.260758+0000","last_active":"2024-04-25T11:00:06.263511+0000","last_peered":"2024-04-25T11:00:06.263511+0000","last_clean":"2024-04-25T11:00:06.263511+0000","last_became_active":"2024-04-25T11:00:06.260455+0000","last_became_peered":"2024-04-25T11:00:06.260455+0000","last_unstale":"2024-04-25T11:00:06.263511+0000","last_undegraded":"2024-04-25T11:00:06.263511+0000","last_fullsized":"2024-04-25T11:00:06.263511+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:19.772176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.252810+0000","last_change":"2024-04-25T11:00:06.252810+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.252810+0000","last_undegraded":"2024-04-25T11:00:06.252810+0000","last_fullsized":"2024-04-25T11:00:06.252810+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.257660+0000","last_change":"2024-04-25T11:00:06.255855+0000","last_active":"2024-04-25T11:00:06.257660+0000","last_peered":"2024-04-25T11:00:06.257660+0000","last_clean":"2024-04-25T11:00:06.257660+0000","last_became_active":"2024-04-25T11:00:06.255509+0000","last_became_peered":"2024-04-25T11:00:06.255509+0000","last_unstale":"2024-04-25T11:00:06.257660+0000","last_undegraded":"2024-04-25T11:00:06.257660+0000","last_fullsized":"2024-04-25T11:00:06.257660+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:24.915258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.244756+0000","last_change":"2024-04-25T11:00:06.244756+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.244756+0000","last_undegraded":"2024-04-25T11:00:06.244756+0000","last_fullsized":"2024-04-25T11:00:06.244756+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.267818+0000","last_change":"2024-04-25T11:00:06.259273+0000","last_active":"2024-04-25T11:00:06.267818+0000","last_peered":"2024-04-25T11:00:06.267818+0000","last_clean":"2024-04-25T11:00:06.267818+0000","last_became_active":"2024-04-25T11:00:06.258923+0000","last_became_peered":"2024-04-25T11:00:06.258923+0000","last_unstale":"2024-04-25T11:00:06.267818+0000","last_undegraded":"2024-04-25T11:00:06.267818+0000","last_fullsized":"2024-04-25T11:00:06.267818+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:11.002336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.250649+0000","last_change":"2024-04-25T11:00:06.250649+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.250649+0000","last_undegraded":"2024-04-25T11:00:06.250649+0000","last_fullsized":"2024-04-25T11:00:06.250649+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.263159+0000","last_change":"2024-04-25T11:00:06.259650+0000","last_active":"2024-04-25T11:00:06.263159+0000","last_peered":"2024-04-25T11:00:06.263159+0000","last_clean":"2024-04-25T11:00:06.263159+0000","last_became_active":"2024-04-25T11:00:06.259152+0000","last_became_peered":"2024-04-25T11:00:06.259152+0000","last_unstale":"2024-04-25T11:00:06.263159+0000","last_undegraded":"2024-04-25T11:00:06.263159+0000","last_fullsized":"2024-04-25T11:00:06.263159+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:00.807719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.251692+0000","last_change":"2024-04-25T11:00:06.251692+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.251692+0000","last_undegraded":"2024-04-25T11:00:06.251692+0000","last_fullsized":"2024-04-25T11:00:06.251692+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.270373+0000","last_change":"2024-04-25T11:00:06.266165+0000","last_active":"2024-04-25T11:00:06.270373+0000","last_peered":"2024-04-25T11:00:06.270373+0000","last_clean":"2024-04-25T11:00:06.270373+0000","last_became_active":"2024-04-25T11:00:06.265773+0000","last_became_peered":"2024-04-25T11:00:06.265773+0000","last_unstale":"2024-04-25T11:00:06.270373+0000","last_undegraded":"2024-04-25T11:00:06.270373+0000","last_fullsized":"2024-04-25T11:00:06.270373+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:55.930698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.248049+0000","last_change":"2024-04-25T11:00:06.248049+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.248049+0000","last_undegraded":"2024-04-25T11:00:06.248049+0000","last_fullsized":"2024-04-25T11:00:06.248049+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.261209+0000","last_change":"2024-04-25T11:00:06.259862+0000","last_active":"2024-04-25T11:00:06.261209+0000","last_peered":"2024-04-25T11:00:06.261209+0000","last_clean":"2024-04-25T11:00:06.261209+0000","last_became_active":"2024-04-25T11:00:06.259570+0000","last_became_peered":"2024-04-25T11:00:06.259570+0000","last_unstale":"2024-04-25T11:00:06.261209+0000","last_undegraded":"2024-04-25T11:00:06.261209+0000","last_fullsized":"2024-04-25T11:00:06.261209+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:37.048525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.247593+0000","last_change":"2024-04-25T11:00:06.247593+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.247593+0000","last_undegraded":"2024-04-25T11:00:06.247593+0000","last_fullsized":"2024-04-25T11:00:06.247593+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.266946+0000","last_change":"2024-04-25T11:00:06.266271+0000","last_active":"2024-04-25T11:00:06.266946+0000","last_peered":"2024-04-25T11:00:06.266946+0000","last_clean":"2024-04-25T11:00:06.266946+0000","last_became_active":"2024-04-25T11:00:06.265815+0000","last_became_peered":"2024-04-25T11:00:06.265815+0000","last_unstale":"2024-04-25T11:00:06.266946+0000","last_undegraded":"2024-04-25T11:00:06.266946+0000","last_fullsized":"2024-04-25T11:00:06.266946+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:45.067082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.253538+0000","last_change":"2024-04-25T11:00:06.253538+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.253538+0000","last_undegraded":"2024-04-25T11:00:06.253538+0000","last_fullsized":"2024-04-25T11:00:06.253538+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.270434+0000","last_change":"2024-04-25T11:00:06.266846+0000","last_active":"2024-04-25T11:00:06.270434+0000","last_peered":"2024-04-25T11:00:06.270434+0000","last_clean":"2024-04-25T11:00:06.270434+0000","last_became_active":"2024-04-25T11:00:06.266553+0000","last_became_peered":"2024-04-25T11:00:06.266553+0000","last_unstale":"2024-04-25T11:00:06.270434+0000","last_undegraded":"2024-04-25T11:00:06.270434+0000","last_fullsized":"2024-04-25T11:00:06.270434+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:02.917078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.250429+0000","last_change":"2024-04-25T11:00:06.250429+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.250429+0000","last_undegraded":"2024-04-25T11:00:06.250429+0000","last_fullsized":"2024-04-25T11:00:06.250429+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.263245+0000","last_change":"2024-04-25T11:00:06.258144+0000","last_active":"2024-04-25T11:00:06.263245+0000","last_peered":"2024-04-25T11:00:06.263245+0000","last_clean":"2024-04-25T11:00:06.263245+0000","last_became_active":"2024-04-25T11:00:06.257775+0000","last_became_peered":"2024-04-25T11:00:06.257775+0000","last_unstale":"2024-04-25T11:00:06.263245+0000","last_undegraded":"2024-04-25T11:00:06.263245+0000","last_fullsized":"2024-04-25T11:00:06.263245+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:06.103278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.247912+0000","last_change":"2024-04-25T11:00:06.247912+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.247912+0000","last_undegraded":"2024-04-25T11:00:06.247912+0000","last_fullsized":"2024-04-25T11:00:06.247912+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.267664+0000","last_change":"2024-04-25T11:00:06.260717+0000","last_active":"2024-04-25T11:00:06.267664+0000","last_peered":"2024-04-25T11:00:06.267664+0000","last_clean":"2024-04-25T11:00:06.267664+0000","last_became_active":"2024-04-25T11:00:06.260433+0000","last_became_peered":"2024-04-25T11:00:06.260433+0000","last_unstale":"2024-04-25T11:00:06.267664+0000","last_undegraded":"2024-04-25T11:00:06.267664+0000","last_fullsized":"2024-04-25T11:00:06.267664+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:11.002336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.261956+0000","last_change":"2024-04-25T11:00:06.261956+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.261956+0000","last_undegraded":"2024-04-25T11:00:06.261956+0000","last_fullsized":"2024-04-25T11:00:06.261956+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.270336+0000","last_change":"2024-04-25T11:00:06.265403+0000","last_active":"2024-04-25T11:00:06.270336+0000","last_peered":"2024-04-25T11:00:06.270336+0000","last_clean":"2024-04-25T11:00:06.270336+0000","last_became_active":"2024-04-25T11:00:06.265062+0000","last_became_peered":"2024-04-25T11:00:06.265062+0000","last_unstale":"2024-04-25T11:00:06.270336+0000","last_undegraded":"2024-04-25T11:00:06.270336+0000","last_fullsized":"2024-04-25T11:00:06.270336+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:37.048525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.244356+0000","last_change":"2024-04-25T11:00:06.244356+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.244356+0000","last_undegraded":"2024-04-25T11:00:06.244356+0000","last_fullsized":"2024-04-25T11:00:06.244356+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.270426+0000","last_change":"2024-04-25T11:00:06.266837+0000","last_active":"2024-04-25T11:00:06.270426+0000","last_peered":"2024-04-25T11:00:06.270426+0000","last_clean":"2024-04-25T11:00:06.270426+0000","last_became_active":"2024-04-25T11:00:06.266003+0000","last_became_peered":"2024-04-25T11:00:06.266003+0000","last_unstale":"2024-04-25T11:00:06.270426+0000","last_undegraded":"2024-04-25T11:00:06.270426+0000","last_fullsized":"2024-04-25T11:00:06.270426+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:42.908127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.245080+0000","last_change":"2024-04-25T11:00:06.245080+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.245080+0000","last_undegraded":"2024-04-25T11:00:06.245080+0000","last_fullsized":"2024-04-25T11:00:06.245080+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.267711+0000","last_change":"2024-04-25T11:00:06.262237+0000","last_active":"2024-04-25T11:00:06.267711+0000","last_peered":"2024-04-25T11:00:06.267711+0000","last_clean":"2024-04-25T11:00:06.267711+0000","last_became_active":"2024-04-25T11:00:06.261872+0000","last_became_peered":"2024-04-25T11:00:06.261872+0000","last_unstale":"2024-04-25T11:00:06.267711+0000","last_undegraded":"2024-04-25T11:00:06.267711+0000","last_fullsized":"2024-04-25T11:00:06.267711+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:08.488342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.254401+0000","last_change":"2024-04-25T11:00:06.254401+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.254401+0000","last_undegraded":"2024-04-25T11:00:06.254401+0000","last_fullsized":"2024-04-25T11:00:06.254401+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.259876+0000","last_change":"2024-04-25T11:00:06.257930+0000","last_active":"2024-04-25T11:00:06.259876+0000","last_peered":"2024-04-25T11:00:06.259876+0000","last_clean":"2024-04-25T11:00:06.259876+0000","last_became_active":"2024-04-25T11:00:06.257326+0000","last_became_peered":"2024-04-25T11:00:06.257326+0000","last_unstale":"2024-04-25T11:00:06.259876+0000","last_undegraded":"2024-04-25T11:00:06.259876+0000","last_fullsized":"2024-04-25T11:00:06.259876+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:38.952429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.251501+0000","last_change":"2024-04-25T11:00:06.251501+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.251501+0000","last_undegraded":"2024-04-25T11:00:06.251501+0000","last_fullsized":"2024-04-25T11:00:06.251501+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.259882+0000","last_change":"2024-04-25T11:00:06.256448+0000","last_active":"2024-04-25T11:00:06.259882+0000","last_peered":"2024-04-25T11:00:06.259882+0000","last_clean":"2024-04-25T11:00:06.259882+0000","last_became_active":"2024-04-25T11:00:06.256095+0000","last_became_peered":"2024-04-25T11:00:06.256095+0000","last_unstale":"2024-04-25T11:00:06.259882+0000","last_undegraded":"2024-04-25T11:00:06.259882+0000","last_fullsized":"2024-04-25T11:00:06.259882+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:04.049748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.250248+0000","last_change":"2024-04-25T11:00:06.250248+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.250248+0000","last_undegraded":"2024-04-25T11:00:06.250248+0000","last_fullsized":"2024-04-25T11:00:06.250248+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.263803+0000","last_change":"2024-04-25T11:00:06.261291+0000","last_active":"2024-04-25T11:00:06.263803+0000","last_peered":"2024-04-25T11:00:06.263803+0000","last_clean":"2024-04-25T11:00:06.263803+0000","last_became_active":"2024-04-25T11:00:06.260969+0000","last_became_peered":"2024-04-25T11:00:06.260969+0000","last_unstale":"2024-04-25T11:00:06.263803+0000","last_undegraded":"2024-04-25T11:00:06.263803+0000","last_fullsized":"2024-04-25T11:00:06.263803+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:31.946740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.247016+0000","last_change":"2024-04-25T11:00:06.247016+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.247016+0000","last_undegraded":"2024-04-25T11:00:06.247016+0000","last_fullsized":"2024-04-25T11:00:06.247016+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.260878+0000","last_change":"2024-04-25T11:00:06.259265+0000","last_active":"2024-04-25T11:00:06.260878+0000","last_peered":"2024-04-25T11:00:06.260878+0000","last_clean":"2024-04-25T11:00:06.260878+0000","last_became_active":"2024-04-25T11:00:06.258949+0000","last_became_peered":"2024-04-25T11:00:06.258949+0000","last_unstale":"2024-04-25T11:00:06.260878+0000","last_undegraded":"2024-04-25T11:00:06.260878+0000","last_fullsized":"2024-04-25T11:00:06.260878+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:08.488342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.249286+0000","last_change":"2024-04-25T11:00:06.249286+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.249286+0000","last_undegraded":"2024-04-25T11:00:06.249286+0000","last_fullsized":"2024-04-25T11:00:06.249286+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.257314+0000","last_change":"2024-04-25T11:00:06.255869+0000","last_active":"2024-04-25T11:00:06.257314+0000","last_peered":"2024-04-25T11:00:06.257314+0000","last_clean":"2024-04-25T11:00:06.257314+0000","last_became_active":"2024-04-25T11:00:06.255468+0000","last_became_peered":"2024-04-25T11:00:06.255468+0000","last_unstale":"2024-04-25T11:00:06.257314+0000","last_undegraded":"2024-04-25T11:00:06.257314+0000","last_fullsized":"2024-04-25T11:00:06.257314+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:40.478099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.246600+0000","last_change":"2024-04-25T11:00:06.246600+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.246600+0000","last_undegraded":"2024-04-25T11:00:06.246600+0000","last_fullsized":"2024-04-25T11:00:06.246600+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.263097+0000","last_change":"2024-04-25T11:00:06.259165+0000","last_active":"2024-04-25T11:00:06.263097+0000","last_peered":"2024-04-25T11:00:06.263097+0000","last_clean":"2024-04-25T11:00:06.263097+0000","last_became_active":"2024-04-25T11:00:06.258814+0000","last_became_peered":"2024-04-25T11:00:06.258814+0000","last_unstale":"2024-04-25T11:00:06.263097+0000","last_undegraded":"2024-04-25T11:00:06.263097+0000","last_fullsized":"2024-04-25T11:00:06.263097+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:46.929021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.261354+0000","last_change":"2024-04-25T11:00:06.261354+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.261354+0000","last_undegraded":"2024-04-25T11:00:06.261354+0000","last_fullsized":"2024-04-25T11:00:06.261354+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.266933+0000","last_change":"2024-04-25T11:00:06.266180+0000","last_active":"2024-04-25T11:00:06.266933+0000","last_peered":"2024-04-25T11:00:06.266933+0000","last_clean":"2024-04-25T11:00:06.266933+0000","last_became_active":"2024-04-25T11:00:06.265801+0000","last_became_peered":"2024-04-25T11:00:06.265801+0000","last_unstale":"2024-04-25T11:00:06.266933+0000","last_undegraded":"2024-04-25T11:00:06.266933+0000","last_fullsized":"2024-04-25T11:00:06.266933+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:27.813350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.245930+0000","last_change":"2024-04-25T11:00:06.245930+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.245930+0000","last_undegraded":"2024-04-25T11:00:06.245930+0000","last_fullsized":"2024-04-25T11:00:06.245930+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.265567+0000","last_change":"2024-04-25T11:00:06.257887+0000","last_active":"2024-04-25T11:00:06.265567+0000","last_peered":"2024-04-25T11:00:06.265567+0000","last_clean":"2024-04-25T11:00:06.265567+0000","last_became_active":"2024-04-25T11:00:06.257563+0000","last_became_peered":"2024-04-25T11:00:06.257563+0000","last_unstale":"2024-04-25T11:00:06.265567+0000","last_undegraded":"2024-04-25T11:00:06.265567+0000","last_fullsized":"2024-04-25T11:00:06.265567+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:44.176669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.253796+0000","last_change":"2024-04-25T11:00:06.253796+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.253796+0000","last_undegraded":"2024-04-25T11:00:06.253796+0000","last_fullsized":"2024-04-25T11:00:06.253796+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.260846+0000","last_change":"2024-04-25T11:00:06.259220+0000","last_active":"2024-04-25T11:00:06.260846+0000","last_peered":"2024-04-25T11:00:06.260846+0000","last_clean":"2024-04-25T11:00:06.260846+0000","last_became_active":"2024-04-25T11:00:06.258895+0000","last_became_peered":"2024-04-25T11:00:06.258895+0000","last_unstale":"2024-04-25T11:00:06.260846+0000","last_undegraded":"2024-04-25T11:00:06.260846+0000","last_fullsized":"2024-04-25T11:00:06.260846+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:11.002336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.248846+0000","last_change":"2024-04-25T11:00:06.248846+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.248846+0000","last_undegraded":"2024-04-25T11:00:06.248846+0000","last_fullsized":"2024-04-25T11:00:06.248846+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.265541+0000","last_change":"2024-04-25T11:00:06.257498+0000","last_active":"2024-04-25T11:00:06.265541+0000","last_peered":"2024-04-25T11:00:06.265541+0000","last_clean":"2024-04-25T11:00:06.265541+0000","last_became_active":"2024-04-25T11:00:06.257083+0000","last_became_peered":"2024-04-25T11:00:06.257083+0000","last_unstale":"2024-04-25T11:00:06.265541+0000","last_undegraded":"2024-04-25T11:00:06.265541+0000","last_fullsized":"2024-04-25T11:00:06.265541+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:30.099257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.245526+0000","last_change":"2024-04-25T11:00:06.245526+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.245526+0000","last_undegraded":"2024-04-25T11:00:06.245526+0000","last_fullsized":"2024-04-25T11:00:06.245526+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.260057+0000","last_change":"2024-04-25T11:00:06.257789+0000","last_active":"2024-04-25T11:00:06.260057+0000","last_peered":"2024-04-25T11:00:06.260057+0000","last_clean":"2024-04-25T11:00:06.260057+0000","last_became_active":"2024-04-25T11:00:06.257450+0000","last_became_peered":"2024-04-25T11:00:06.257450+0000","last_unstale":"2024-04-25T11:00:06.260057+0000","last_undegraded":"2024-04-25T11:00:06.260057+0000","last_fullsized":"2024-04-25T11:00:06.260057+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:44.176669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.248002+0000","last_change":"2024-04-25T11:00:06.248002+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.248002+0000","last_undegraded":"2024-04-25T11:00:06.248002+0000","last_fullsized":"2024-04-25T11:00:06.248002+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.265641+0000","last_change":"2024-04-25T11:00:06.257884+0000","last_active":"2024-04-25T11:00:06.265641+0000","last_peered":"2024-04-25T11:00:06.265641+0000","last_clean":"2024-04-25T11:00:06.265641+0000","last_became_active":"2024-04-25T11:00:06.257586+0000","last_became_peered":"2024-04-25T11:00:06.257586+0000","last_unstale":"2024-04-25T11:00:06.265641+0000","last_undegraded":"2024-04-25T11:00:06.265641+0000","last_fullsized":"2024-04-25T11:00:06.265641+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:40.478099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.248465+0000","last_change":"2024-04-25T11:00:06.248465+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.248465+0000","last_undegraded":"2024-04-25T11:00:06.248465+0000","last_fullsized":"2024-04-25T11:00:06.248465+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.263074+0000","last_change":"2024-04-25T11:00:06.258316+0000","last_active":"2024-04-25T11:00:06.263074+0000","last_peered":"2024-04-25T11:00:06.263074+0000","last_clean":"2024-04-25T11:00:06.263074+0000","last_became_active":"2024-04-25T11:00:06.257838+0000","last_became_peered":"2024-04-25T11:00:06.257838+0000","last_unstale":"2024-04-25T11:00:06.263074+0000","last_undegraded":"2024-04-25T11:00:06.263074+0000","last_fullsized":"2024-04-25T11:00:06.263074+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:19.075396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.247389+0000","last_change":"2024-04-25T11:00:06.247389+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.247389+0000","last_undegraded":"2024-04-25T11:00:06.247389+0000","last_fullsized":"2024-04-25T11:00:06.247389+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.270359+0000","last_change":"2024-04-25T11:00:06.266105+0000","last_active":"2024-04-25T11:00:06.270359+0000","last_peered":"2024-04-25T11:00:06.270359+0000","last_clean":"2024-04-25T11:00:06.270359+0000","last_became_active":"2024-04-25T11:00:06.265786+0000","last_became_peered":"2024-04-25T11:00:06.265786+0000","last_unstale":"2024-04-25T11:00:06.270359+0000","last_undegraded":"2024-04-25T11:00:06.270359+0000","last_fullsized":"2024-04-25T11:00:06.270359+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:03.333540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.257208+0000","last_change":"2024-04-25T11:00:06.257208+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.257208+0000","last_undegraded":"2024-04-25T11:00:06.257208+0000","last_fullsized":"2024-04-25T11:00:06.257208+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.256715+0000","last_change":"2024-04-25T11:00:06.250003+0000","last_active":"2024-04-25T11:00:06.256715+0000","last_peered":"2024-04-25T11:00:06.256715+0000","last_clean":"2024-04-25T11:00:06.256715+0000","last_became_active":"2024-04-25T11:00:06.249632+0000","last_became_peered":"2024-04-25T11:00:06.249632+0000","last_unstale":"2024-04-25T11:00:06.256715+0000","last_undegraded":"2024-04-25T11:00:06.256715+0000","last_fullsized":"2024-04-25T11:00:06.256715+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:30.099257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":125,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.263800+0000","last_change":"2024-04-25T10:59:07.839466+0000","last_active":"2024-04-25T11:00:06.263800+0000","last_peered":"2024-04-25T11:00:06.263800+0000","last_clean":"2024-04-25T11:00:06.263800+0000","last_became_active":"2024-04-25T10:59:07.836269+0000","last_became_peered":"2024-04-25T10:59:07.836269+0000","last_unstale":"2024-04-25T11:00:06.263800+0000","last_undegraded":"2024-04-25T11:00:06.263800+0000","last_fullsized":"2024-04-25T11:00:06.263800+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:06.549663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:06.549663+0000","last_clean_scrub_stamp":"2024-04-25T10:59:06.549663+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:21.575330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":905216,"data_stored":705476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":115108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":115108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":115108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":115108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":115108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":905216,"data_stored":705476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:00:11.618 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:00:11.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.760+0000 7f81c4a51640 1 Processor -- start 2024-04-25T11:00:11.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.760+0000 7f81c4a51640 1 -- start start 2024-04-25T11:00:11.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.762+0000 7f81c4a51640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f81c00cf770 0x7f81c0162590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:11.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.762+0000 7f81c4a51640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f81c0162ad0 0x7f81c00b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:11.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.762+0000 7f81c4a51640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81c00cfc30 con 0x7f81c00b91d0 2024-04-25T11:00:11.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.762+0000 7f81c4a51640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f81c00d1ff0 con 0x7f81c0162ad0 2024-04-25T11:00:11.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.762+0000 7f81c4a51640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f81c00d2890 con 0x7f81c00cf770 2024-04-25T11:00:11.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.762+0000 7f81bed9d640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f81c0162ad0 0x7f81c00b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:11.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.762+0000 7f81bed9d640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f81c0162ad0 0x7f81c00b0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36590/0 (socket says 172.21.15.69:36590) 2024-04-25T11:00:11.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.762+0000 7f81bed9d640 1 -- 172.21.15.69:0/2913055399 learned_addr learned my addr 172.21.15.69:0/2913055399 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:11.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.762+0000 7f81b7fff640 1 --2- 172.21.15.69:0/2913055399 >> v2:172.21.15.146:3300/0 conn(0x7f81c00cf770 0x7f81c0162590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:11.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.762+0000 7f81bed9d640 1 -- 172.21.15.69:0/2913055399 >> v2:172.21.15.146:3300/0 conn(0x7f81c00cf770 msgr2=0x7f81c0162590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:11.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.762+0000 7f81bed9d640 1 --2- 172.21.15.69:0/2913055399 >> v2:172.21.15.146:3300/0 conn(0x7f81c00cf770 0x7f81c0162590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:11.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.762+0000 7f81bed9d640 1 -- 172.21.15.69:0/2913055399 >> v1:172.21.15.146:6789/0 conn(0x7f81c00b91d0 legacy=0x7f81c0161e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:11.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.762+0000 7f81bed9d640 1 -- 172.21.15.69:0/2913055399 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c00d2240 con 0x7f81c0162ad0 2024-04-25T11:00:11.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.763+0000 7f81bed9d640 1 --2- 172.21.15.69:0/2913055399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81c0162ad0 0x7f81c00b0690 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f81c0075fa0 tx=0x7f81b802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc7dffccf68a810 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:11.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.763+0000 7f81bdd9b640 1 -- 172.21.15.69:0/2913055399 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b8036070 con 0x7f81c0162ad0 2024-04-25T11:00:11.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.763+0000 7f81bdd9b640 1 -- 172.21.15.69:0/2913055399 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81b8031600 con 0x7f81c0162ad0 2024-04-25T11:00:11.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.763+0000 7f81bdd9b640 1 -- 172.21.15.69:0/2913055399 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b80319d0 con 0x7f81c0162ad0 2024-04-25T11:00:11.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.764+0000 7f81c4a51640 1 -- 172.21.15.69:0/2913055399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81c0162ad0 msgr2=0x7f81c00b0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:11.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.764+0000 7f81c4a51640 1 --2- 172.21.15.69:0/2913055399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81c0162ad0 0x7f81c00b0690 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f81c0075fa0 tx=0x7f81b802a660 comp rx=0 tx=0).stop 2024-04-25T11:00:11.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.764+0000 7f81c4a51640 1 -- 172.21.15.69:0/2913055399 shutdown_connections 2024-04-25T11:00:11.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.764+0000 7f81c4a51640 1 --2- 172.21.15.69:0/2913055399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81c0162ad0 0x7f81c00b0690 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:11.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.764+0000 7f81c4a51640 1 --2- 172.21.15.69:0/2913055399 >> v2:172.21.15.146:3300/0 conn(0x7f81c00cf770 0x7f81c0162590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:11.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.764+0000 7f81c4a51640 1 -- 172.21.15.69:0/2913055399 >> 172.21.15.69:0/2913055399 conn(0x7f81c00d3aa0 msgr2=0x7f81c00d3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:11.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.764+0000 7f81c4a51640 1 -- 172.21.15.69:0/2913055399 shutdown_connections 2024-04-25T11:00:11.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.764+0000 7f81c4a51640 1 -- 172.21.15.69:0/2913055399 wait complete. 2024-04-25T11:00:11.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.765+0000 7f81c4a51640 1 Processor -- start 2024-04-25T11:00:11.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.766+0000 7f81c4a51640 1 -- start start 2024-04-25T11:00:11.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.767+0000 7f81c4a51640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81c00b91d0 0x7f81c01572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:11.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.767+0000 7f81c4a51640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81c0162ad0 0x7f81c0156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:11.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.767+0000 7f81c4a51640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81c0160060 0x7f81c0160470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:11.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.767+0000 7f81c4a51640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f81c00b24a0 con 0x7f81c0160060 2024-04-25T11:00:11.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.767+0000 7f81c4a51640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f81c00b2320 con 0x7f81c00b91d0 2024-04-25T11:00:11.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.767+0000 7f81c4a51640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f81c00b0bd0 con 0x7f81c0162ad0 2024-04-25T11:00:11.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.767+0000 7f81bed9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81c0160060 0x7f81c0160470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:11.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.767+0000 7f81bed9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81c0160060 0x7f81c0160470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36600/0 (socket says 172.21.15.69:36600) 2024-04-25T11:00:11.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.767+0000 7f81bed9d640 1 -- 172.21.15.69:0/2290420391 learned_addr learned my addr 172.21.15.69:0/2290420391 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:11.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.767+0000 7f81be59c640 1 --2- 172.21.15.69:0/2290420391 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81c00b91d0 0x7f81c01572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:11.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.767+0000 7f81b7fff640 1 --2- 172.21.15.69:0/2290420391 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81c0162ad0 0x7f81c0156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:11.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.767+0000 7f81bed9d640 1 -- 172.21.15.69:0/2290420391 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81c0162ad0 msgr2=0x7f81c0156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:11.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.767+0000 7f81bed9d640 1 --2- 172.21.15.69:0/2290420391 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81c0162ad0 0x7f81c0156bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:11.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.767+0000 7f81bed9d640 1 -- 172.21.15.69:0/2290420391 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81c00b91d0 msgr2=0x7f81c01572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:11.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.767+0000 7f81bed9d640 1 --2- 172.21.15.69:0/2290420391 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81c00b91d0 0x7f81c01572a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:11.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.767+0000 7f81bed9d640 1 -- 172.21.15.69:0/2290420391 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c0160d30 con 0x7f81c0160060 2024-04-25T11:00:11.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.767+0000 7f81bed9d640 1 --2- 172.21.15.69:0/2290420391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81c0160060 0x7f81c0160470 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f81b8034040 tx=0x7f81b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:11.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.768+0000 7f81b6ffd640 1 -- 172.21.15.69:0/2290420391 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b8036050 con 0x7f81c0160060 2024-04-25T11:00:11.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.768+0000 7f81b6ffd640 1 -- 172.21.15.69:0/2290420391 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81b8042070 con 0x7f81c0160060 2024-04-25T11:00:11.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.768+0000 7f81c4a51640 1 -- 172.21.15.69:0/2290420391 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c0161010 con 0x7f81c0160060 2024-04-25T11:00:11.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.768+0000 7f81b6ffd640 1 -- 172.21.15.69:0/2290420391 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b8009510 con 0x7f81c0160060 2024-04-25T11:00:11.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.768+0000 7f81c4a51640 1 -- 172.21.15.69:0/2290420391 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c00cbb60 con 0x7f81c0160060 2024-04-25T11:00:11.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.769+0000 7f81c4a51640 1 -- 172.21.15.69:0/2290420391 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81c0050740 con 0x7f81c0160060 2024-04-25T11:00:11.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.769+0000 7f81b6ffd640 1 -- 172.21.15.69:0/2290420391 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f81b8007bd0 con 0x7f81c0160060 2024-04-25T11:00:11.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.770+0000 7f81b6ffd640 1 --2- 172.21.15.69:0/2290420391 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f81a00744e0 0x7f81a00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:11.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.770+0000 7f81b6ffd640 1 -- 172.21.15.69:0/2290420391 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f81b80b60e0 con 0x7f81c0160060 2024-04-25T11:00:11.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.770+0000 7f81be59c640 1 --2- 172.21.15.69:0/2290420391 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f81a00744e0 0x7f81a00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:11.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.770+0000 7f81be59c640 1 --2- 172.21.15.69:0/2290420391 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f81a00744e0 0x7f81a00769f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f81b0006fd0 tx=0x7f81b0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:11.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.775+0000 7f81b6ffd640 1 -- 172.21.15.69:0/2290420391 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f81b8080870 con 0x7f81c0160060 2024-04-25T11:00:11.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.944+0000 7f81c4a51640 1 -- 172.21.15.69:0/2290420391 --> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f81c00cbe00 con 0x7f81a00744e0 2024-04-25T11:00:11.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.950+0000 7f81b6ffd640 1 -- 172.21.15.69:0/2290420391 <== mgr.4107 v2:172.21.15.69:6832/1107421267 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246618 (secure 0 0 0) 0x7f81c00cbe00 con 0x7f81a00744e0 2024-04-25T11:00:11.953 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:00:11.954 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-04-25T11:00:11.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.953+0000 7f81c4a51640 1 -- 172.21.15.69:0/2290420391 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f81a00744e0 msgr2=0x7f81a00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:11.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.953+0000 7f81c4a51640 1 --2- 172.21.15.69:0/2290420391 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f81a00744e0 0x7f81a00769f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f81b0006fd0 tx=0x7f81b0008040 comp rx=0 tx=0).stop 2024-04-25T11:00:11.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.953+0000 7f81c4a51640 1 -- 172.21.15.69:0/2290420391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81c0160060 msgr2=0x7f81c0160470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:11.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.953+0000 7f81c4a51640 1 --2- 172.21.15.69:0/2290420391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81c0160060 0x7f81c0160470 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f81b8034040 tx=0x7f81b8004a40 comp rx=0 tx=0).stop 2024-04-25T11:00:11.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.953+0000 7f81c4a51640 1 -- 172.21.15.69:0/2290420391 shutdown_connections 2024-04-25T11:00:11.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.953+0000 7f81c4a51640 1 --2- 172.21.15.69:0/2290420391 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f81a00744e0 0x7f81a00769f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:11.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.953+0000 7f81c4a51640 1 --2- 172.21.15.69:0/2290420391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81c0160060 0x7f81c0160470 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:11.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.953+0000 7f81c4a51640 1 --2- 172.21.15.69:0/2290420391 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81c0162ad0 0x7f81c0156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:11.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.953+0000 7f81c4a51640 1 --2- 172.21.15.69:0/2290420391 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81c00b91d0 0x7f81c01572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:11.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.953+0000 7f81c4a51640 1 -- 172.21.15.69:0/2290420391 >> 172.21.15.69:0/2290420391 conn(0x7f81c00d3aa0 msgr2=0x7f81c015dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:11.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.953+0000 7f81c4a51640 1 -- 172.21.15.69:0/2290420391 shutdown_connections 2024-04-25T11:00:11.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:11.953+0000 7f81c4a51640 1 -- 172.21.15.69:0/2290420391 wait complete. 2024-04-25T11:00:11.965 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":56,"stamp":"2024-04-25T11:00:10.416036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219768,"kb_used_data":3604,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754952,"statfs":{"total":773094113280,"available":772869070848,"internally_reserved":0,"allocated":3690496,"data_stored":2107867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.181025"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.247019+0000","last_change":"2024-04-25T11:00:06.247019+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.247019+0000","last_undegraded":"2024-04-25T11:00:06.247019+0000","last_fullsized":"2024-04-25T11:00:06.247019+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.248970+0000","last_change":"2024-04-25T11:00:06.248970+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.248970+0000","last_undegraded":"2024-04-25T11:00:06.248970+0000","last_fullsized":"2024-04-25T11:00:06.248970+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.253027+0000","last_change":"2024-04-25T11:00:06.253027+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.253027+0000","last_undegraded":"2024-04-25T11:00:06.253027+0000","last_fullsized":"2024-04-25T11:00:06.253027+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.253669+0000","last_change":"2024-04-25T11:00:06.253669+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.253669+0000","last_undegraded":"2024-04-25T11:00:06.253669+0000","last_fullsized":"2024-04-25T11:00:06.253669+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.252482+0000","last_change":"2024-04-25T11:00:06.252482+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.252482+0000","last_undegraded":"2024-04-25T11:00:06.252482+0000","last_fullsized":"2024-04-25T11:00:06.252482+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.248650+0000","last_change":"2024-04-25T11:00:06.248650+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.248650+0000","last_undegraded":"2024-04-25T11:00:06.248650+0000","last_fullsized":"2024-04-25T11:00:06.248650+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.259535+0000","last_change":"2024-04-25T11:00:06.259535+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.259535+0000","last_undegraded":"2024-04-25T11:00:06.259535+0000","last_fullsized":"2024-04-25T11:00:06.259535+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.263557+0000","last_change":"2024-04-25T11:00:06.263557+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.263557+0000","last_undegraded":"2024-04-25T11:00:06.263557+0000","last_fullsized":"2024-04-25T11:00:06.263557+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.246492+0000","last_change":"2024-04-25T11:00:06.246492+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.246492+0000","last_undegraded":"2024-04-25T11:00:06.246492+0000","last_fullsized":"2024-04-25T11:00:06.246492+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.251364+0000","last_change":"2024-04-25T11:00:06.251364+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.251364+0000","last_undegraded":"2024-04-25T11:00:06.251364+0000","last_fullsized":"2024-04-25T11:00:06.251364+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.260217+0000","last_change":"2024-04-25T11:00:06.260217+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.260217+0000","last_undegraded":"2024-04-25T11:00:06.260217+0000","last_fullsized":"2024-04-25T11:00:06.260217+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.252165+0000","last_change":"2024-04-25T11:00:06.252165+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.252165+0000","last_undegraded":"2024-04-25T11:00:06.252165+0000","last_fullsized":"2024-04-25T11:00:06.252165+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.251519+0000","last_change":"2024-04-25T11:00:06.251519+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.251519+0000","last_undegraded":"2024-04-25T11:00:06.251519+0000","last_fullsized":"2024-04-25T11:00:06.251519+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.248054+0000","last_change":"2024-04-25T11:00:06.248054+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.248054+0000","last_undegraded":"2024-04-25T11:00:06.248054+0000","last_fullsized":"2024-04-25T11:00:06.248054+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.252810+0000","last_change":"2024-04-25T11:00:06.252810+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.252810+0000","last_undegraded":"2024-04-25T11:00:06.252810+0000","last_fullsized":"2024-04-25T11:00:06.252810+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.257769+0000","last_change":"2024-04-25T11:00:06.257769+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.257769+0000","last_undegraded":"2024-04-25T11:00:06.257769+0000","last_fullsized":"2024-04-25T11:00:06.257769+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.261835+0000","last_change":"2024-04-25T11:00:06.261835+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.261835+0000","last_undegraded":"2024-04-25T11:00:06.261835+0000","last_fullsized":"2024-04-25T11:00:06.261835+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.247792+0000","last_change":"2024-04-25T11:00:06.247792+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.247792+0000","last_undegraded":"2024-04-25T11:00:06.247792+0000","last_fullsized":"2024-04-25T11:00:06.247792+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.247704+0000","last_change":"2024-04-25T11:00:06.247704+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.247704+0000","last_undegraded":"2024-04-25T11:00:06.247704+0000","last_fullsized":"2024-04-25T11:00:06.247704+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.251408+0000","last_change":"2024-04-25T11:00:06.251408+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.251408+0000","last_undegraded":"2024-04-25T11:00:06.251408+0000","last_fullsized":"2024-04-25T11:00:06.251408+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.247701+0000","last_change":"2024-04-25T11:00:06.247701+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.247701+0000","last_undegraded":"2024-04-25T11:00:06.247701+0000","last_fullsized":"2024-04-25T11:00:06.247701+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.244357+0000","last_change":"2024-04-25T11:00:06.244357+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.244357+0000","last_undegraded":"2024-04-25T11:00:06.244357+0000","last_fullsized":"2024-04-25T11:00:06.244357+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.249829+0000","last_change":"2024-04-25T11:00:06.249829+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.249829+0000","last_undegraded":"2024-04-25T11:00:06.249829+0000","last_fullsized":"2024-04-25T11:00:06.249829+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.251978+0000","last_change":"2024-04-25T11:00:06.251978+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.251978+0000","last_undegraded":"2024-04-25T11:00:06.251978+0000","last_fullsized":"2024-04-25T11:00:06.251978+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.249905+0000","last_change":"2024-04-25T11:00:06.249905+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.249905+0000","last_undegraded":"2024-04-25T11:00:06.249905+0000","last_fullsized":"2024-04-25T11:00:06.249905+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.247067+0000","last_change":"2024-04-25T11:00:06.247067+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.247067+0000","last_undegraded":"2024-04-25T11:00:06.247067+0000","last_fullsized":"2024-04-25T11:00:06.247067+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.250240+0000","last_change":"2024-04-25T11:00:06.250240+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.250240+0000","last_undegraded":"2024-04-25T11:00:06.250240+0000","last_fullsized":"2024-04-25T11:00:06.250240+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.251501+0000","last_change":"2024-04-25T11:00:06.251501+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.251501+0000","last_undegraded":"2024-04-25T11:00:06.251501+0000","last_fullsized":"2024-04-25T11:00:06.251501+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.246143+0000","last_change":"2024-04-25T11:00:06.246143+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.246143+0000","last_undegraded":"2024-04-25T11:00:06.246143+0000","last_fullsized":"2024-04-25T11:00:06.246143+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.259514+0000","last_change":"2024-04-25T11:00:06.259514+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.259514+0000","last_undegraded":"2024-04-25T11:00:06.259514+0000","last_fullsized":"2024-04-25T11:00:06.259514+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.244943+0000","last_change":"2024-04-25T11:00:06.244943+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.244943+0000","last_undegraded":"2024-04-25T11:00:06.244943+0000","last_fullsized":"2024-04-25T11:00:06.244943+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.263045+0000","last_change":"2024-04-25T11:00:06.263045+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.263045+0000","last_undegraded":"2024-04-25T11:00:06.263045+0000","last_fullsized":"2024-04-25T11:00:06.263045+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.248830+0000","last_change":"2024-04-25T11:00:06.248830+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.248830+0000","last_undegraded":"2024-04-25T11:00:06.248830+0000","last_fullsized":"2024-04-25T11:00:06.248830+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.262835+0000","last_change":"2024-04-25T11:00:06.257849+0000","last_active":"2024-04-25T11:00:06.262835+0000","last_peered":"2024-04-25T11:00:06.262835+0000","last_clean":"2024-04-25T11:00:06.262835+0000","last_became_active":"2024-04-25T11:00:06.257452+0000","last_became_peered":"2024-04-25T11:00:06.257452+0000","last_unstale":"2024-04-25T11:00:06.262835+0000","last_undegraded":"2024-04-25T11:00:06.262835+0000","last_fullsized":"2024-04-25T11:00:06.262835+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:44.176669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.246458+0000","last_change":"2024-04-25T11:00:06.246458+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.246458+0000","last_undegraded":"2024-04-25T11:00:06.246458+0000","last_fullsized":"2024-04-25T11:00:06.246458+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.266419+0000","last_change":"2024-04-25T11:00:06.265699+0000","last_active":"2024-04-25T11:00:06.266419+0000","last_peered":"2024-04-25T11:00:06.266419+0000","last_clean":"2024-04-25T11:00:06.266419+0000","last_became_active":"2024-04-25T11:00:06.265112+0000","last_became_peered":"2024-04-25T11:00:06.265112+0000","last_unstale":"2024-04-25T11:00:06.266419+0000","last_undegraded":"2024-04-25T11:00:06.266419+0000","last_fullsized":"2024-04-25T11:00:06.266419+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:03.333540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.246789+0000","last_change":"2024-04-25T11:00:06.246789+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.246789+0000","last_undegraded":"2024-04-25T11:00:06.246789+0000","last_fullsized":"2024-04-25T11:00:06.246789+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.267960+0000","last_change":"2024-04-25T11:00:06.262272+0000","last_active":"2024-04-25T11:00:06.267960+0000","last_peered":"2024-04-25T11:00:06.267960+0000","last_clean":"2024-04-25T11:00:06.267960+0000","last_became_active":"2024-04-25T11:00:06.261982+0000","last_became_peered":"2024-04-25T11:00:06.261982+0000","last_unstale":"2024-04-25T11:00:06.267960+0000","last_undegraded":"2024-04-25T11:00:06.267960+0000","last_fullsized":"2024-04-25T11:00:06.267960+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:08.488342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.249670+0000","last_change":"2024-04-25T11:00:06.249670+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.249670+0000","last_undegraded":"2024-04-25T11:00:06.249670+0000","last_fullsized":"2024-04-25T11:00:06.249670+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.260566+0000","last_change":"2024-04-25T11:00:06.257693+0000","last_active":"2024-04-25T11:00:06.260566+0000","last_peered":"2024-04-25T11:00:06.260566+0000","last_clean":"2024-04-25T11:00:06.260566+0000","last_became_active":"2024-04-25T11:00:06.257402+0000","last_became_peered":"2024-04-25T11:00:06.257402+0000","last_unstale":"2024-04-25T11:00:06.260566+0000","last_undegraded":"2024-04-25T11:00:06.260566+0000","last_fullsized":"2024-04-25T11:00:06.260566+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:19.772176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.257968+0000","last_change":"2024-04-25T11:00:06.257968+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.257968+0000","last_undegraded":"2024-04-25T11:00:06.257968+0000","last_fullsized":"2024-04-25T11:00:06.257968+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.263511+0000","last_change":"2024-04-25T11:00:06.260758+0000","last_active":"2024-04-25T11:00:06.263511+0000","last_peered":"2024-04-25T11:00:06.263511+0000","last_clean":"2024-04-25T11:00:06.263511+0000","last_became_active":"2024-04-25T11:00:06.260455+0000","last_became_peered":"2024-04-25T11:00:06.260455+0000","last_unstale":"2024-04-25T11:00:06.263511+0000","last_undegraded":"2024-04-25T11:00:06.263511+0000","last_fullsized":"2024-04-25T11:00:06.263511+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:19.772176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.252810+0000","last_change":"2024-04-25T11:00:06.252810+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.252810+0000","last_undegraded":"2024-04-25T11:00:06.252810+0000","last_fullsized":"2024-04-25T11:00:06.252810+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.257660+0000","last_change":"2024-04-25T11:00:06.255855+0000","last_active":"2024-04-25T11:00:06.257660+0000","last_peered":"2024-04-25T11:00:06.257660+0000","last_clean":"2024-04-25T11:00:06.257660+0000","last_became_active":"2024-04-25T11:00:06.255509+0000","last_became_peered":"2024-04-25T11:00:06.255509+0000","last_unstale":"2024-04-25T11:00:06.257660+0000","last_undegraded":"2024-04-25T11:00:06.257660+0000","last_fullsized":"2024-04-25T11:00:06.257660+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:24.915258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.244756+0000","last_change":"2024-04-25T11:00:06.244756+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.244756+0000","last_undegraded":"2024-04-25T11:00:06.244756+0000","last_fullsized":"2024-04-25T11:00:06.244756+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.267818+0000","last_change":"2024-04-25T11:00:06.259273+0000","last_active":"2024-04-25T11:00:06.267818+0000","last_peered":"2024-04-25T11:00:06.267818+0000","last_clean":"2024-04-25T11:00:06.267818+0000","last_became_active":"2024-04-25T11:00:06.258923+0000","last_became_peered":"2024-04-25T11:00:06.258923+0000","last_unstale":"2024-04-25T11:00:06.267818+0000","last_undegraded":"2024-04-25T11:00:06.267818+0000","last_fullsized":"2024-04-25T11:00:06.267818+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:11.002336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.250649+0000","last_change":"2024-04-25T11:00:06.250649+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.250649+0000","last_undegraded":"2024-04-25T11:00:06.250649+0000","last_fullsized":"2024-04-25T11:00:06.250649+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.263159+0000","last_change":"2024-04-25T11:00:06.259650+0000","last_active":"2024-04-25T11:00:06.263159+0000","last_peered":"2024-04-25T11:00:06.263159+0000","last_clean":"2024-04-25T11:00:06.263159+0000","last_became_active":"2024-04-25T11:00:06.259152+0000","last_became_peered":"2024-04-25T11:00:06.259152+0000","last_unstale":"2024-04-25T11:00:06.263159+0000","last_undegraded":"2024-04-25T11:00:06.263159+0000","last_fullsized":"2024-04-25T11:00:06.263159+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:00.807719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.251692+0000","last_change":"2024-04-25T11:00:06.251692+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.251692+0000","last_undegraded":"2024-04-25T11:00:06.251692+0000","last_fullsized":"2024-04-25T11:00:06.251692+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.270373+0000","last_change":"2024-04-25T11:00:06.266165+0000","last_active":"2024-04-25T11:00:06.270373+0000","last_peered":"2024-04-25T11:00:06.270373+0000","last_clean":"2024-04-25T11:00:06.270373+0000","last_became_active":"2024-04-25T11:00:06.265773+0000","last_became_peered":"2024-04-25T11:00:06.265773+0000","last_unstale":"2024-04-25T11:00:06.270373+0000","last_undegraded":"2024-04-25T11:00:06.270373+0000","last_fullsized":"2024-04-25T11:00:06.270373+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:55.930698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.248049+0000","last_change":"2024-04-25T11:00:06.248049+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.248049+0000","last_undegraded":"2024-04-25T11:00:06.248049+0000","last_fullsized":"2024-04-25T11:00:06.248049+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.261209+0000","last_change":"2024-04-25T11:00:06.259862+0000","last_active":"2024-04-25T11:00:06.261209+0000","last_peered":"2024-04-25T11:00:06.261209+0000","last_clean":"2024-04-25T11:00:06.261209+0000","last_became_active":"2024-04-25T11:00:06.259570+0000","last_became_peered":"2024-04-25T11:00:06.259570+0000","last_unstale":"2024-04-25T11:00:06.261209+0000","last_undegraded":"2024-04-25T11:00:06.261209+0000","last_fullsized":"2024-04-25T11:00:06.261209+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:37.048525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.247593+0000","last_change":"2024-04-25T11:00:06.247593+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.247593+0000","last_undegraded":"2024-04-25T11:00:06.247593+0000","last_fullsized":"2024-04-25T11:00:06.247593+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.266946+0000","last_change":"2024-04-25T11:00:06.266271+0000","last_active":"2024-04-25T11:00:06.266946+0000","last_peered":"2024-04-25T11:00:06.266946+0000","last_clean":"2024-04-25T11:00:06.266946+0000","last_became_active":"2024-04-25T11:00:06.265815+0000","last_became_peered":"2024-04-25T11:00:06.265815+0000","last_unstale":"2024-04-25T11:00:06.266946+0000","last_undegraded":"2024-04-25T11:00:06.266946+0000","last_fullsized":"2024-04-25T11:00:06.266946+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:45.067082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.253538+0000","last_change":"2024-04-25T11:00:06.253538+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.253538+0000","last_undegraded":"2024-04-25T11:00:06.253538+0000","last_fullsized":"2024-04-25T11:00:06.253538+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.270434+0000","last_change":"2024-04-25T11:00:06.266846+0000","last_active":"2024-04-25T11:00:06.270434+0000","last_peered":"2024-04-25T11:00:06.270434+0000","last_clean":"2024-04-25T11:00:06.270434+0000","last_became_active":"2024-04-25T11:00:06.266553+0000","last_became_peered":"2024-04-25T11:00:06.266553+0000","last_unstale":"2024-04-25T11:00:06.270434+0000","last_undegraded":"2024-04-25T11:00:06.270434+0000","last_fullsized":"2024-04-25T11:00:06.270434+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:02.917078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.250429+0000","last_change":"2024-04-25T11:00:06.250429+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.250429+0000","last_undegraded":"2024-04-25T11:00:06.250429+0000","last_fullsized":"2024-04-25T11:00:06.250429+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.263245+0000","last_change":"2024-04-25T11:00:06.258144+0000","last_active":"2024-04-25T11:00:06.263245+0000","last_peered":"2024-04-25T11:00:06.263245+0000","last_clean":"2024-04-25T11:00:06.263245+0000","last_became_active":"2024-04-25T11:00:06.257775+0000","last_became_peered":"2024-04-25T11:00:06.257775+0000","last_unstale":"2024-04-25T11:00:06.263245+0000","last_undegraded":"2024-04-25T11:00:06.263245+0000","last_fullsized":"2024-04-25T11:00:06.263245+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:06.103278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.247912+0000","last_change":"2024-04-25T11:00:06.247912+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.247912+0000","last_undegraded":"2024-04-25T11:00:06.247912+0000","last_fullsized":"2024-04-25T11:00:06.247912+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.267664+0000","last_change":"2024-04-25T11:00:06.260717+0000","last_active":"2024-04-25T11:00:06.267664+0000","last_peered":"2024-04-25T11:00:06.267664+0000","last_clean":"2024-04-25T11:00:06.267664+0000","last_became_active":"2024-04-25T11:00:06.260433+0000","last_became_peered":"2024-04-25T11:00:06.260433+0000","last_unstale":"2024-04-25T11:00:06.267664+0000","last_undegraded":"2024-04-25T11:00:06.267664+0000","last_fullsized":"2024-04-25T11:00:06.267664+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:11.002336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.261956+0000","last_change":"2024-04-25T11:00:06.261956+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.261956+0000","last_undegraded":"2024-04-25T11:00:06.261956+0000","last_fullsized":"2024-04-25T11:00:06.261956+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.270336+0000","last_change":"2024-04-25T11:00:06.265403+0000","last_active":"2024-04-25T11:00:06.270336+0000","last_peered":"2024-04-25T11:00:06.270336+0000","last_clean":"2024-04-25T11:00:06.270336+0000","last_became_active":"2024-04-25T11:00:06.265062+0000","last_became_peered":"2024-04-25T11:00:06.265062+0000","last_unstale":"2024-04-25T11:00:06.270336+0000","last_undegraded":"2024-04-25T11:00:06.270336+0000","last_fullsized":"2024-04-25T11:00:06.270336+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:37.048525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.244356+0000","last_change":"2024-04-25T11:00:06.244356+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.244356+0000","last_undegraded":"2024-04-25T11:00:06.244356+0000","last_fullsized":"2024-04-25T11:00:06.244356+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.270426+0000","last_change":"2024-04-25T11:00:06.266837+0000","last_active":"2024-04-25T11:00:06.270426+0000","last_peered":"2024-04-25T11:00:06.270426+0000","last_clean":"2024-04-25T11:00:06.270426+0000","last_became_active":"2024-04-25T11:00:06.266003+0000","last_became_peered":"2024-04-25T11:00:06.266003+0000","last_unstale":"2024-04-25T11:00:06.270426+0000","last_undegraded":"2024-04-25T11:00:06.270426+0000","last_fullsized":"2024-04-25T11:00:06.270426+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:42.908127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.245080+0000","last_change":"2024-04-25T11:00:06.245080+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.245080+0000","last_undegraded":"2024-04-25T11:00:06.245080+0000","last_fullsized":"2024-04-25T11:00:06.245080+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.267711+0000","last_change":"2024-04-25T11:00:06.262237+0000","last_active":"2024-04-25T11:00:06.267711+0000","last_peered":"2024-04-25T11:00:06.267711+0000","last_clean":"2024-04-25T11:00:06.267711+0000","last_became_active":"2024-04-25T11:00:06.261872+0000","last_became_peered":"2024-04-25T11:00:06.261872+0000","last_unstale":"2024-04-25T11:00:06.267711+0000","last_undegraded":"2024-04-25T11:00:06.267711+0000","last_fullsized":"2024-04-25T11:00:06.267711+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:08.488342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.254401+0000","last_change":"2024-04-25T11:00:06.254401+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.254401+0000","last_undegraded":"2024-04-25T11:00:06.254401+0000","last_fullsized":"2024-04-25T11:00:06.254401+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.259876+0000","last_change":"2024-04-25T11:00:06.257930+0000","last_active":"2024-04-25T11:00:06.259876+0000","last_peered":"2024-04-25T11:00:06.259876+0000","last_clean":"2024-04-25T11:00:06.259876+0000","last_became_active":"2024-04-25T11:00:06.257326+0000","last_became_peered":"2024-04-25T11:00:06.257326+0000","last_unstale":"2024-04-25T11:00:06.259876+0000","last_undegraded":"2024-04-25T11:00:06.259876+0000","last_fullsized":"2024-04-25T11:00:06.259876+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:38.952429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.251501+0000","last_change":"2024-04-25T11:00:06.251501+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.251501+0000","last_undegraded":"2024-04-25T11:00:06.251501+0000","last_fullsized":"2024-04-25T11:00:06.251501+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.259882+0000","last_change":"2024-04-25T11:00:06.256448+0000","last_active":"2024-04-25T11:00:06.259882+0000","last_peered":"2024-04-25T11:00:06.259882+0000","last_clean":"2024-04-25T11:00:06.259882+0000","last_became_active":"2024-04-25T11:00:06.256095+0000","last_became_peered":"2024-04-25T11:00:06.256095+0000","last_unstale":"2024-04-25T11:00:06.259882+0000","last_undegraded":"2024-04-25T11:00:06.259882+0000","last_fullsized":"2024-04-25T11:00:06.259882+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:04.049748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.250248+0000","last_change":"2024-04-25T11:00:06.250248+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.250248+0000","last_undegraded":"2024-04-25T11:00:06.250248+0000","last_fullsized":"2024-04-25T11:00:06.250248+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.263803+0000","last_change":"2024-04-25T11:00:06.261291+0000","last_active":"2024-04-25T11:00:06.263803+0000","last_peered":"2024-04-25T11:00:06.263803+0000","last_clean":"2024-04-25T11:00:06.263803+0000","last_became_active":"2024-04-25T11:00:06.260969+0000","last_became_peered":"2024-04-25T11:00:06.260969+0000","last_unstale":"2024-04-25T11:00:06.263803+0000","last_undegraded":"2024-04-25T11:00:06.263803+0000","last_fullsized":"2024-04-25T11:00:06.263803+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:31.946740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.247016+0000","last_change":"2024-04-25T11:00:06.247016+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.247016+0000","last_undegraded":"2024-04-25T11:00:06.247016+0000","last_fullsized":"2024-04-25T11:00:06.247016+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.260878+0000","last_change":"2024-04-25T11:00:06.259265+0000","last_active":"2024-04-25T11:00:06.260878+0000","last_peered":"2024-04-25T11:00:06.260878+0000","last_clean":"2024-04-25T11:00:06.260878+0000","last_became_active":"2024-04-25T11:00:06.258949+0000","last_became_peered":"2024-04-25T11:00:06.258949+0000","last_unstale":"2024-04-25T11:00:06.260878+0000","last_undegraded":"2024-04-25T11:00:06.260878+0000","last_fullsized":"2024-04-25T11:00:06.260878+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:08.488342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.249286+0000","last_change":"2024-04-25T11:00:06.249286+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.249286+0000","last_undegraded":"2024-04-25T11:00:06.249286+0000","last_fullsized":"2024-04-25T11:00:06.249286+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.257314+0000","last_change":"2024-04-25T11:00:06.255869+0000","last_active":"2024-04-25T11:00:06.257314+0000","last_peered":"2024-04-25T11:00:06.257314+0000","last_clean":"2024-04-25T11:00:06.257314+0000","last_became_active":"2024-04-25T11:00:06.255468+0000","last_became_peered":"2024-04-25T11:00:06.255468+0000","last_unstale":"2024-04-25T11:00:06.257314+0000","last_undegraded":"2024-04-25T11:00:06.257314+0000","last_fullsized":"2024-04-25T11:00:06.257314+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:40.478099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.246600+0000","last_change":"2024-04-25T11:00:06.246600+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.246600+0000","last_undegraded":"2024-04-25T11:00:06.246600+0000","last_fullsized":"2024-04-25T11:00:06.246600+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.263097+0000","last_change":"2024-04-25T11:00:06.259165+0000","last_active":"2024-04-25T11:00:06.263097+0000","last_peered":"2024-04-25T11:00:06.263097+0000","last_clean":"2024-04-25T11:00:06.263097+0000","last_became_active":"2024-04-25T11:00:06.258814+0000","last_became_peered":"2024-04-25T11:00:06.258814+0000","last_unstale":"2024-04-25T11:00:06.263097+0000","last_undegraded":"2024-04-25T11:00:06.263097+0000","last_fullsized":"2024-04-25T11:00:06.263097+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:46.929021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.261354+0000","last_change":"2024-04-25T11:00:06.261354+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.261354+0000","last_undegraded":"2024-04-25T11:00:06.261354+0000","last_fullsized":"2024-04-25T11:00:06.261354+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.266933+0000","last_change":"2024-04-25T11:00:06.266180+0000","last_active":"2024-04-25T11:00:06.266933+0000","last_peered":"2024-04-25T11:00:06.266933+0000","last_clean":"2024-04-25T11:00:06.266933+0000","last_became_active":"2024-04-25T11:00:06.265801+0000","last_became_peered":"2024-04-25T11:00:06.265801+0000","last_unstale":"2024-04-25T11:00:06.266933+0000","last_undegraded":"2024-04-25T11:00:06.266933+0000","last_fullsized":"2024-04-25T11:00:06.266933+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:27.813350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.245930+0000","last_change":"2024-04-25T11:00:06.245930+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.245930+0000","last_undegraded":"2024-04-25T11:00:06.245930+0000","last_fullsized":"2024-04-25T11:00:06.245930+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.265567+0000","last_change":"2024-04-25T11:00:06.257887+0000","last_active":"2024-04-25T11:00:06.265567+0000","last_peered":"2024-04-25T11:00:06.265567+0000","last_clean":"2024-04-25T11:00:06.265567+0000","last_became_active":"2024-04-25T11:00:06.257563+0000","last_became_peered":"2024-04-25T11:00:06.257563+0000","last_unstale":"2024-04-25T11:00:06.265567+0000","last_undegraded":"2024-04-25T11:00:06.265567+0000","last_fullsized":"2024-04-25T11:00:06.265567+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:44.176669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.253796+0000","last_change":"2024-04-25T11:00:06.253796+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.253796+0000","last_undegraded":"2024-04-25T11:00:06.253796+0000","last_fullsized":"2024-04-25T11:00:06.253796+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.260846+0000","last_change":"2024-04-25T11:00:06.259220+0000","last_active":"2024-04-25T11:00:06.260846+0000","last_peered":"2024-04-25T11:00:06.260846+0000","last_clean":"2024-04-25T11:00:06.260846+0000","last_became_active":"2024-04-25T11:00:06.258895+0000","last_became_peered":"2024-04-25T11:00:06.258895+0000","last_unstale":"2024-04-25T11:00:06.260846+0000","last_undegraded":"2024-04-25T11:00:06.260846+0000","last_fullsized":"2024-04-25T11:00:06.260846+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:11.002336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.248846+0000","last_change":"2024-04-25T11:00:06.248846+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.248846+0000","last_undegraded":"2024-04-25T11:00:06.248846+0000","last_fullsized":"2024-04-25T11:00:06.248846+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.265541+0000","last_change":"2024-04-25T11:00:06.257498+0000","last_active":"2024-04-25T11:00:06.265541+0000","last_peered":"2024-04-25T11:00:06.265541+0000","last_clean":"2024-04-25T11:00:06.265541+0000","last_became_active":"2024-04-25T11:00:06.257083+0000","last_became_peered":"2024-04-25T11:00:06.257083+0000","last_unstale":"2024-04-25T11:00:06.265541+0000","last_undegraded":"2024-04-25T11:00:06.265541+0000","last_fullsized":"2024-04-25T11:00:06.265541+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:30.099257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.245526+0000","last_change":"2024-04-25T11:00:06.245526+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.245526+0000","last_undegraded":"2024-04-25T11:00:06.245526+0000","last_fullsized":"2024-04-25T11:00:06.245526+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.260057+0000","last_change":"2024-04-25T11:00:06.257789+0000","last_active":"2024-04-25T11:00:06.260057+0000","last_peered":"2024-04-25T11:00:06.260057+0000","last_clean":"2024-04-25T11:00:06.260057+0000","last_became_active":"2024-04-25T11:00:06.257450+0000","last_became_peered":"2024-04-25T11:00:06.257450+0000","last_unstale":"2024-04-25T11:00:06.260057+0000","last_undegraded":"2024-04-25T11:00:06.260057+0000","last_fullsized":"2024-04-25T11:00:06.260057+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:44.176669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.248002+0000","last_change":"2024-04-25T11:00:06.248002+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.248002+0000","last_undegraded":"2024-04-25T11:00:06.248002+0000","last_fullsized":"2024-04-25T11:00:06.248002+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.265641+0000","last_change":"2024-04-25T11:00:06.257884+0000","last_active":"2024-04-25T11:00:06.265641+0000","last_peered":"2024-04-25T11:00:06.265641+0000","last_clean":"2024-04-25T11:00:06.265641+0000","last_became_active":"2024-04-25T11:00:06.257586+0000","last_became_peered":"2024-04-25T11:00:06.257586+0000","last_unstale":"2024-04-25T11:00:06.265641+0000","last_undegraded":"2024-04-25T11:00:06.265641+0000","last_fullsized":"2024-04-25T11:00:06.265641+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:40.478099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.248465+0000","last_change":"2024-04-25T11:00:06.248465+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.248465+0000","last_undegraded":"2024-04-25T11:00:06.248465+0000","last_fullsized":"2024-04-25T11:00:06.248465+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.263074+0000","last_change":"2024-04-25T11:00:06.258316+0000","last_active":"2024-04-25T11:00:06.263074+0000","last_peered":"2024-04-25T11:00:06.263074+0000","last_clean":"2024-04-25T11:00:06.263074+0000","last_became_active":"2024-04-25T11:00:06.257838+0000","last_became_peered":"2024-04-25T11:00:06.257838+0000","last_unstale":"2024-04-25T11:00:06.263074+0000","last_undegraded":"2024-04-25T11:00:06.263074+0000","last_fullsized":"2024-04-25T11:00:06.263074+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:19.075396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.247389+0000","last_change":"2024-04-25T11:00:06.247389+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.247389+0000","last_undegraded":"2024-04-25T11:00:06.247389+0000","last_fullsized":"2024-04-25T11:00:06.247389+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.270359+0000","last_change":"2024-04-25T11:00:06.266105+0000","last_active":"2024-04-25T11:00:06.270359+0000","last_peered":"2024-04-25T11:00:06.270359+0000","last_clean":"2024-04-25T11:00:06.270359+0000","last_became_active":"2024-04-25T11:00:06.265786+0000","last_became_peered":"2024-04-25T11:00:06.265786+0000","last_unstale":"2024-04-25T11:00:06.270359+0000","last_undegraded":"2024-04-25T11:00:06.270359+0000","last_fullsized":"2024-04-25T11:00:06.270359+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:03.333540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-25T11:00:06.257208+0000","last_change":"2024-04-25T11:00:06.257208+0000","last_active":"2024-04-25T11:00:06.235114+0000","last_peered":"2024-04-25T11:00:06.235114+0000","last_clean":"2024-04-25T11:00:06.235114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:00:06.257208+0000","last_undegraded":"2024-04-25T11:00:06.257208+0000","last_fullsized":"2024-04-25T11:00:06.257208+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.256715+0000","last_change":"2024-04-25T11:00:06.250003+0000","last_active":"2024-04-25T11:00:06.256715+0000","last_peered":"2024-04-25T11:00:06.256715+0000","last_clean":"2024-04-25T11:00:06.256715+0000","last_became_active":"2024-04-25T11:00:06.249632+0000","last_became_peered":"2024-04-25T11:00:06.249632+0000","last_unstale":"2024-04-25T11:00:06.256715+0000","last_undegraded":"2024-04-25T11:00:06.256715+0000","last_fullsized":"2024-04-25T11:00:06.256715+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:30.099257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":125,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T11:00:06.263800+0000","last_change":"2024-04-25T10:59:07.839466+0000","last_active":"2024-04-25T11:00:06.263800+0000","last_peered":"2024-04-25T11:00:06.263800+0000","last_clean":"2024-04-25T11:00:06.263800+0000","last_became_active":"2024-04-25T10:59:07.836269+0000","last_became_peered":"2024-04-25T10:59:07.836269+0000","last_unstale":"2024-04-25T11:00:06.263800+0000","last_undegraded":"2024-04-25T11:00:06.263800+0000","last_fullsized":"2024-04-25T11:00:06.263800+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:06.549663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:06.549663+0000","last_clean_scrub_stamp":"2024-04-25T10:59:06.549663+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:21.575330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":905216,"data_stored":705476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":115108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":115108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":115108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":115108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":115108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":905216,"data_stored":705476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:00:11.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2024-04-25T11:00:11.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.247019+0000', 'last_change': '2024-04-25T11:00:06.247019+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.247019+0000', 'last_undegraded': '2024-04-25T11:00:06.247019+0000', 'last_fullsized': '2024-04-25T11:00:06.247019+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T11:00:11.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2024-04-25T11:00:11.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.248970+0000', 'last_change': '2024-04-25T11:00:06.248970+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.248970+0000', 'last_undegraded': '2024-04-25T11:00:06.248970+0000', 'last_fullsized': '2024-04-25T11:00:06.248970+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T11:00:11.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2024-04-25T11:00:11.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.253027+0000', 'last_change': '2024-04-25T11:00:06.253027+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.253027+0000', 'last_undegraded': '2024-04-25T11:00:06.253027+0000', 'last_fullsized': '2024-04-25T11:00:06.253027+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:00:11.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2024-04-25T11:00:11.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.253669+0000', 'last_change': '2024-04-25T11:00:06.253669+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.253669+0000', 'last_undegraded': '2024-04-25T11:00:06.253669+0000', 'last_fullsized': '2024-04-25T11:00:06.253669+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T11:00:11.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2024-04-25T11:00:11.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.252482+0000', 'last_change': '2024-04-25T11:00:06.252482+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.252482+0000', 'last_undegraded': '2024-04-25T11:00:06.252482+0000', 'last_fullsized': '2024-04-25T11:00:06.252482+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T11:00:11.970 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2024-04-25T11:00:11.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.248650+0000', 'last_change': '2024-04-25T11:00:06.248650+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.248650+0000', 'last_undegraded': '2024-04-25T11:00:06.248650+0000', 'last_fullsized': '2024-04-25T11:00:06.248650+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T11:00:11.970 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2024-04-25T11:00:11.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.259535+0000', 'last_change': '2024-04-25T11:00:06.259535+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.259535+0000', 'last_undegraded': '2024-04-25T11:00:06.259535+0000', 'last_fullsized': '2024-04-25T11:00:06.259535+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T11:00:11.970 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2024-04-25T11:00:11.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.263557+0000', 'last_change': '2024-04-25T11:00:06.263557+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.263557+0000', 'last_undegraded': '2024-04-25T11:00:06.263557+0000', 'last_fullsized': '2024-04-25T11:00:06.263557+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T11:00:11.970 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2024-04-25T11:00:11.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.246492+0000', 'last_change': '2024-04-25T11:00:06.246492+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.246492+0000', 'last_undegraded': '2024-04-25T11:00:06.246492+0000', 'last_fullsized': '2024-04-25T11:00:06.246492+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T11:00:11.970 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2024-04-25T11:00:11.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.251364+0000', 'last_change': '2024-04-25T11:00:06.251364+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.251364+0000', 'last_undegraded': '2024-04-25T11:00:06.251364+0000', 'last_fullsized': '2024-04-25T11:00:06.251364+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T11:00:11.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2024-04-25T11:00:11.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.260217+0000', 'last_change': '2024-04-25T11:00:06.260217+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.260217+0000', 'last_undegraded': '2024-04-25T11:00:06.260217+0000', 'last_fullsized': '2024-04-25T11:00:06.260217+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T11:00:11.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2024-04-25T11:00:11.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.252165+0000', 'last_change': '2024-04-25T11:00:06.252165+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.252165+0000', 'last_undegraded': '2024-04-25T11:00:06.252165+0000', 'last_fullsized': '2024-04-25T11:00:06.252165+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:00:11.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2024-04-25T11:00:11.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.251519+0000', 'last_change': '2024-04-25T11:00:06.251519+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.251519+0000', 'last_undegraded': '2024-04-25T11:00:06.251519+0000', 'last_fullsized': '2024-04-25T11:00:06.251519+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T11:00:11.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2024-04-25T11:00:11.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.248054+0000', 'last_change': '2024-04-25T11:00:06.248054+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.248054+0000', 'last_undegraded': '2024-04-25T11:00:06.248054+0000', 'last_fullsized': '2024-04-25T11:00:06.248054+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T11:00:11.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2024-04-25T11:00:11.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.252810+0000', 'last_change': '2024-04-25T11:00:06.252810+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.252810+0000', 'last_undegraded': '2024-04-25T11:00:06.252810+0000', 'last_fullsized': '2024-04-25T11:00:06.252810+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:00:11.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2024-04-25T11:00:11.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.257769+0000', 'last_change': '2024-04-25T11:00:06.257769+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.257769+0000', 'last_undegraded': '2024-04-25T11:00:06.257769+0000', 'last_fullsized': '2024-04-25T11:00:06.257769+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T11:00:11.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2024-04-25T11:00:11.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.261835+0000', 'last_change': '2024-04-25T11:00:06.261835+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.261835+0000', 'last_undegraded': '2024-04-25T11:00:06.261835+0000', 'last_fullsized': '2024-04-25T11:00:06.261835+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T11:00:11.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2024-04-25T11:00:11.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.247792+0000', 'last_change': '2024-04-25T11:00:06.247792+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.247792+0000', 'last_undegraded': '2024-04-25T11:00:06.247792+0000', 'last_fullsized': '2024-04-25T11:00:06.247792+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T11:00:11.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2024-04-25T11:00:11.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.247704+0000', 'last_change': '2024-04-25T11:00:06.247704+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.247704+0000', 'last_undegraded': '2024-04-25T11:00:06.247704+0000', 'last_fullsized': '2024-04-25T11:00:06.247704+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T11:00:11.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2024-04-25T11:00:11.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.251408+0000', 'last_change': '2024-04-25T11:00:06.251408+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.251408+0000', 'last_undegraded': '2024-04-25T11:00:06.251408+0000', 'last_fullsized': '2024-04-25T11:00:06.251408+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T11:00:11.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2024-04-25T11:00:11.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.247701+0000', 'last_change': '2024-04-25T11:00:06.247701+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.247701+0000', 'last_undegraded': '2024-04-25T11:00:06.247701+0000', 'last_fullsized': '2024-04-25T11:00:06.247701+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T11:00:11.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2024-04-25T11:00:11.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.244357+0000', 'last_change': '2024-04-25T11:00:06.244357+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.244357+0000', 'last_undegraded': '2024-04-25T11:00:06.244357+0000', 'last_fullsized': '2024-04-25T11:00:06.244357+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T11:00:11.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2024-04-25T11:00:11.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.249829+0000', 'last_change': '2024-04-25T11:00:06.249829+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.249829+0000', 'last_undegraded': '2024-04-25T11:00:06.249829+0000', 'last_fullsized': '2024-04-25T11:00:06.249829+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T11:00:11.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2024-04-25T11:00:11.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.251978+0000', 'last_change': '2024-04-25T11:00:06.251978+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.251978+0000', 'last_undegraded': '2024-04-25T11:00:06.251978+0000', 'last_fullsized': '2024-04-25T11:00:06.251978+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:00:11.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2024-04-25T11:00:11.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.249905+0000', 'last_change': '2024-04-25T11:00:06.249905+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.249905+0000', 'last_undegraded': '2024-04-25T11:00:06.249905+0000', 'last_fullsized': '2024-04-25T11:00:06.249905+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T11:00:11.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2024-04-25T11:00:11.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.247067+0000', 'last_change': '2024-04-25T11:00:06.247067+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.247067+0000', 'last_undegraded': '2024-04-25T11:00:06.247067+0000', 'last_fullsized': '2024-04-25T11:00:06.247067+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T11:00:11.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2024-04-25T11:00:11.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.250240+0000', 'last_change': '2024-04-25T11:00:06.250240+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.250240+0000', 'last_undegraded': '2024-04-25T11:00:06.250240+0000', 'last_fullsized': '2024-04-25T11:00:06.250240+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T11:00:11.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2024-04-25T11:00:11.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.251501+0000', 'last_change': '2024-04-25T11:00:06.251501+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.251501+0000', 'last_undegraded': '2024-04-25T11:00:06.251501+0000', 'last_fullsized': '2024-04-25T11:00:06.251501+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:00:11.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2024-04-25T11:00:11.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.246143+0000', 'last_change': '2024-04-25T11:00:06.246143+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.246143+0000', 'last_undegraded': '2024-04-25T11:00:06.246143+0000', 'last_fullsized': '2024-04-25T11:00:06.246143+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T11:00:11.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2024-04-25T11:00:11.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.259514+0000', 'last_change': '2024-04-25T11:00:06.259514+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.259514+0000', 'last_undegraded': '2024-04-25T11:00:06.259514+0000', 'last_fullsized': '2024-04-25T11:00:06.259514+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T11:00:11.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2024-04-25T11:00:11.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.244943+0000', 'last_change': '2024-04-25T11:00:06.244943+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.244943+0000', 'last_undegraded': '2024-04-25T11:00:06.244943+0000', 'last_fullsized': '2024-04-25T11:00:06.244943+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T11:00:11.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2024-04-25T11:00:11.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.263045+0000', 'last_change': '2024-04-25T11:00:06.263045+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.263045+0000', 'last_undegraded': '2024-04-25T11:00:06.263045+0000', 'last_fullsized': '2024-04-25T11:00:06.263045+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T11:00:11.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2024-04-25T11:00:11.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.248830+0000', 'last_change': '2024-04-25T11:00:06.248830+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.248830+0000', 'last_undegraded': '2024-04-25T11:00:06.248830+0000', 'last_fullsized': '2024-04-25T11:00:06.248830+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T11:00:11.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2024-04-25T11:00:11.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.246458+0000', 'last_change': '2024-04-25T11:00:06.246458+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.246458+0000', 'last_undegraded': '2024-04-25T11:00:06.246458+0000', 'last_fullsized': '2024-04-25T11:00:06.246458+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T11:00:11.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2024-04-25T11:00:11.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.246789+0000', 'last_change': '2024-04-25T11:00:06.246789+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.246789+0000', 'last_undegraded': '2024-04-25T11:00:06.246789+0000', 'last_fullsized': '2024-04-25T11:00:06.246789+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T11:00:11.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2024-04-25T11:00:11.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.249670+0000', 'last_change': '2024-04-25T11:00:06.249670+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.249670+0000', 'last_undegraded': '2024-04-25T11:00:06.249670+0000', 'last_fullsized': '2024-04-25T11:00:06.249670+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T11:00:11.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2024-04-25T11:00:11.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.257968+0000', 'last_change': '2024-04-25T11:00:06.257968+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.257968+0000', 'last_undegraded': '2024-04-25T11:00:06.257968+0000', 'last_fullsized': '2024-04-25T11:00:06.257968+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T11:00:11.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2024-04-25T11:00:11.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.252810+0000', 'last_change': '2024-04-25T11:00:06.252810+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.252810+0000', 'last_undegraded': '2024-04-25T11:00:06.252810+0000', 'last_fullsized': '2024-04-25T11:00:06.252810+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:00:11.976 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2024-04-25T11:00:11.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.244756+0000', 'last_change': '2024-04-25T11:00:06.244756+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.244756+0000', 'last_undegraded': '2024-04-25T11:00:06.244756+0000', 'last_fullsized': '2024-04-25T11:00:06.244756+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T11:00:11.976 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2024-04-25T11:00:11.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.250649+0000', 'last_change': '2024-04-25T11:00:06.250649+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.250649+0000', 'last_undegraded': '2024-04-25T11:00:06.250649+0000', 'last_fullsized': '2024-04-25T11:00:06.250649+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T11:00:11.976 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2024-04-25T11:00:11.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.251692+0000', 'last_change': '2024-04-25T11:00:06.251692+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.251692+0000', 'last_undegraded': '2024-04-25T11:00:06.251692+0000', 'last_fullsized': '2024-04-25T11:00:06.251692+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:00:11.976 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2024-04-25T11:00:11.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.248049+0000', 'last_change': '2024-04-25T11:00:06.248049+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.248049+0000', 'last_undegraded': '2024-04-25T11:00:06.248049+0000', 'last_fullsized': '2024-04-25T11:00:06.248049+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T11:00:11.976 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2024-04-25T11:00:11.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.247593+0000', 'last_change': '2024-04-25T11:00:06.247593+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.247593+0000', 'last_undegraded': '2024-04-25T11:00:06.247593+0000', 'last_fullsized': '2024-04-25T11:00:06.247593+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T11:00:11.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-04-25T11:00:11.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.253538+0000', 'last_change': '2024-04-25T11:00:06.253538+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.253538+0000', 'last_undegraded': '2024-04-25T11:00:06.253538+0000', 'last_fullsized': '2024-04-25T11:00:06.253538+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:00:11.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-04-25T11:00:11.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.250429+0000', 'last_change': '2024-04-25T11:00:06.250429+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.250429+0000', 'last_undegraded': '2024-04-25T11:00:06.250429+0000', 'last_fullsized': '2024-04-25T11:00:06.250429+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T11:00:11.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2024-04-25T11:00:11.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.247912+0000', 'last_change': '2024-04-25T11:00:06.247912+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.247912+0000', 'last_undegraded': '2024-04-25T11:00:06.247912+0000', 'last_fullsized': '2024-04-25T11:00:06.247912+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T11:00:11.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2024-04-25T11:00:11.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.261956+0000', 'last_change': '2024-04-25T11:00:06.261956+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.261956+0000', 'last_undegraded': '2024-04-25T11:00:06.261956+0000', 'last_fullsized': '2024-04-25T11:00:06.261956+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T11:00:11.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2024-04-25T11:00:11.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.244356+0000', 'last_change': '2024-04-25T11:00:06.244356+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.244356+0000', 'last_undegraded': '2024-04-25T11:00:06.244356+0000', 'last_fullsized': '2024-04-25T11:00:06.244356+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T11:00:11.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2024-04-25T11:00:11.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.245080+0000', 'last_change': '2024-04-25T11:00:06.245080+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.245080+0000', 'last_undegraded': '2024-04-25T11:00:06.245080+0000', 'last_fullsized': '2024-04-25T11:00:06.245080+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T11:00:11.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2024-04-25T11:00:11.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.254401+0000', 'last_change': '2024-04-25T11:00:06.254401+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.254401+0000', 'last_undegraded': '2024-04-25T11:00:06.254401+0000', 'last_fullsized': '2024-04-25T11:00:06.254401+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:00:11.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-04-25T11:00:11.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.251501+0000', 'last_change': '2024-04-25T11:00:06.251501+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.251501+0000', 'last_undegraded': '2024-04-25T11:00:06.251501+0000', 'last_fullsized': '2024-04-25T11:00:06.251501+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T11:00:11.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2024-04-25T11:00:11.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.250248+0000', 'last_change': '2024-04-25T11:00:06.250248+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.250248+0000', 'last_undegraded': '2024-04-25T11:00:06.250248+0000', 'last_fullsized': '2024-04-25T11:00:06.250248+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T11:00:11.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2024-04-25T11:00:11.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.247016+0000', 'last_change': '2024-04-25T11:00:06.247016+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.247016+0000', 'last_undegraded': '2024-04-25T11:00:06.247016+0000', 'last_fullsized': '2024-04-25T11:00:06.247016+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T11:00:11.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-04-25T11:00:11.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.249286+0000', 'last_change': '2024-04-25T11:00:06.249286+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.249286+0000', 'last_undegraded': '2024-04-25T11:00:06.249286+0000', 'last_fullsized': '2024-04-25T11:00:06.249286+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T11:00:11.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2024-04-25T11:00:11.979 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.246600+0000', 'last_change': '2024-04-25T11:00:06.246600+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.246600+0000', 'last_undegraded': '2024-04-25T11:00:06.246600+0000', 'last_fullsized': '2024-04-25T11:00:06.246600+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T11:00:11.979 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2024-04-25T11:00:11.979 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.261354+0000', 'last_change': '2024-04-25T11:00:06.261354+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.261354+0000', 'last_undegraded': '2024-04-25T11:00:06.261354+0000', 'last_fullsized': '2024-04-25T11:00:06.261354+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T11:00:11.979 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2024-04-25T11:00:11.979 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.245930+0000', 'last_change': '2024-04-25T11:00:06.245930+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.245930+0000', 'last_undegraded': '2024-04-25T11:00:06.245930+0000', 'last_fullsized': '2024-04-25T11:00:06.245930+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T11:00:11.979 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2024-04-25T11:00:11.979 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.253796+0000', 'last_change': '2024-04-25T11:00:06.253796+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.253796+0000', 'last_undegraded': '2024-04-25T11:00:06.253796+0000', 'last_fullsized': '2024-04-25T11:00:06.253796+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:00:11.979 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2024-04-25T11:00:11.979 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.248846+0000', 'last_change': '2024-04-25T11:00:06.248846+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.248846+0000', 'last_undegraded': '2024-04-25T11:00:06.248846+0000', 'last_fullsized': '2024-04-25T11:00:06.248846+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T11:00:11.979 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2024-04-25T11:00:11.979 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.245526+0000', 'last_change': '2024-04-25T11:00:06.245526+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.245526+0000', 'last_undegraded': '2024-04-25T11:00:06.245526+0000', 'last_fullsized': '2024-04-25T11:00:06.245526+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T11:00:11.980 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2024-04-25T11:00:11.980 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.248002+0000', 'last_change': '2024-04-25T11:00:06.248002+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.248002+0000', 'last_undegraded': '2024-04-25T11:00:06.248002+0000', 'last_fullsized': '2024-04-25T11:00:06.248002+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T11:00:11.980 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2024-04-25T11:00:11.980 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.248465+0000', 'last_change': '2024-04-25T11:00:06.248465+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.248465+0000', 'last_undegraded': '2024-04-25T11:00:06.248465+0000', 'last_fullsized': '2024-04-25T11:00:06.248465+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T11:00:11.980 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2024-04-25T11:00:11.980 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.247389+0000', 'last_change': '2024-04-25T11:00:06.247389+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.247389+0000', 'last_undegraded': '2024-04-25T11:00:06.247389+0000', 'last_fullsized': '2024-04-25T11:00:06.247389+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T11:00:11.980 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2024-04-25T11:00:11.980 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:00:06.257208+0000', 'last_change': '2024-04-25T11:00:06.257208+0000', 'last_active': '2024-04-25T11:00:06.235114+0000', 'last_peered': '2024-04-25T11:00:06.235114+0000', 'last_clean': '2024-04-25T11:00:06.235114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:00:06.257208+0000', 'last_undegraded': '2024-04-25T11:00:06.257208+0000', 'last_fullsized': '2024-04-25T11:00:06.257208+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'last_clean_scrub_stamp': '2024-04-25T11:00:06.235114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T11:00:11.981 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T11:00:12.074 INFO:tasks.ceph.mds.a.smithi069.stderr:2024-04-25T11:00:12.073+0000 7fca9755b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:00:12.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.111+0000 7f516eee0640 1 Processor -- start 2024-04-25T11:00:12.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.111+0000 7f516eee0640 1 -- start start 2024-04-25T11:00:12.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.112+0000 7f516eee0640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f51680d4570 0x7f51680b4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:12.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.112+0000 7f516eee0640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f51680b2dc0 0x7f51680b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:12.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.112+0000 7f516eee0640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f516806d870 con 0x7f516806e8f0 2024-04-25T11:00:12.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.112+0000 7f516eee0640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f516806e460 con 0x7f51680d4570 2024-04-25T11:00:12.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.112+0000 7f516eee0640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f516806db10 con 0x7f51680b2dc0 2024-04-25T11:00:12.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.113+0000 7f516cc7c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f51680d4570 0x7f51680b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:12.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.113+0000 7f516cc7c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f51680d4570 0x7f51680b4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36608/0 (socket says 172.21.15.69:36608) 2024-04-25T11:00:12.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.113+0000 7f516cc7c640 1 -- 172.21.15.69:0/2971603317 learned_addr learned my addr 172.21.15.69:0/2971603317 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:12.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.113+0000 7f516d47d640 1 --2- 172.21.15.69:0/2971603317 >> v2:172.21.15.146:3300/0 conn(0x7f51680b2dc0 0x7f51680b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:12.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.113+0000 7f516cc7c640 1 -- 172.21.15.69:0/2971603317 >> v2:172.21.15.146:3300/0 conn(0x7f51680b2dc0 msgr2=0x7f51680b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:12.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.113+0000 7f516cc7c640 1 --2- 172.21.15.69:0/2971603317 >> v2:172.21.15.146:3300/0 conn(0x7f51680b2dc0 0x7f51680b31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:12.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.113+0000 7f516cc7c640 1 -- 172.21.15.69:0/2971603317 >> v1:172.21.15.146:6789/0 conn(0x7f516806e8f0 legacy=0x7f51680d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:12.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.113+0000 7f516cc7c640 1 -- 172.21.15.69:0/2971603317 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516806e0f0 con 0x7f51680d4570 2024-04-25T11:00:12.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.114+0000 7f516cc7c640 1 --2- 172.21.15.69:0/2971603317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51680d4570 0x7f51680b4830 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f516806e5e0 tx=0x7f515802a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c6f6b565c59dc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:12.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.114+0000 7f51677fe640 1 -- 172.21.15.69:0/2971603317 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5158004030 con 0x7f51680d4570 2024-04-25T11:00:12.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.114+0000 7f51677fe640 1 -- 172.21.15.69:0/2971603317 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51580041d0 con 0x7f51680d4570 2024-04-25T11:00:12.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.114+0000 7f51677fe640 1 -- 172.21.15.69:0/2971603317 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5158003610 con 0x7f51680d4570 2024-04-25T11:00:12.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.114+0000 7f516eee0640 1 -- 172.21.15.69:0/2971603317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51680d4570 msgr2=0x7f51680b4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:12.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.114+0000 7f516eee0640 1 --2- 172.21.15.69:0/2971603317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51680d4570 0x7f51680b4830 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f516806e5e0 tx=0x7f515802a160 comp rx=0 tx=0).stop 2024-04-25T11:00:12.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.114+0000 7f516eee0640 1 -- 172.21.15.69:0/2971603317 shutdown_connections 2024-04-25T11:00:12.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.114+0000 7f516eee0640 1 --2- 172.21.15.69:0/2971603317 >> v2:172.21.15.146:3300/0 conn(0x7f51680b2dc0 0x7f51680b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:12.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.114+0000 7f516eee0640 1 --2- 172.21.15.69:0/2971603317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51680d4570 0x7f51680b4830 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:12.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.115+0000 7f516eee0640 1 -- 172.21.15.69:0/2971603317 >> 172.21.15.69:0/2971603317 conn(0x7f51680d3a50 msgr2=0x7f51680d3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:12.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.115+0000 7f516eee0640 1 -- 172.21.15.69:0/2971603317 shutdown_connections 2024-04-25T11:00:12.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.115+0000 7f516eee0640 1 -- 172.21.15.69:0/2971603317 wait complete. 2024-04-25T11:00:12.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.116+0000 7f516eee0640 1 Processor -- start 2024-04-25T11:00:12.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.116+0000 7f516eee0640 1 -- start start 2024-04-25T11:00:12.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.118+0000 7f516eee0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f516806e8f0 0x7f51681f67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:12.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.118+0000 7f516eee0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51680b2dc0 0x7f51681f9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:12.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.118+0000 7f516eee0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51680d4570 0x7f51680cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:12.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.118+0000 7f516eee0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f516806ed00 con 0x7f516806e8f0 2024-04-25T11:00:12.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.118+0000 7f516eee0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f51680d0fb0 con 0x7f51680b2dc0 2024-04-25T11:00:12.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.118+0000 7f516eee0640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f51680b3710 con 0x7f51680d4570 2024-04-25T11:00:12.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.118+0000 7f516cc7c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f516806e8f0 0x7f51681f67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:12.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.118+0000 7f516cc7c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f516806e8f0 0x7f51681f67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36620/0 (socket says 172.21.15.69:36620) 2024-04-25T11:00:12.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.118+0000 7f516cc7c640 1 -- 172.21.15.69:0/1000632713 learned_addr learned my addr 172.21.15.69:0/1000632713 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:12.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.118+0000 7f516d47d640 1 --2- 172.21.15.69:0/1000632713 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51680d4570 0x7f51680cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:12.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.118+0000 7f5167fff640 1 --2- 172.21.15.69:0/1000632713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51680b2dc0 0x7f51681f9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:12.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.118+0000 7f516cc7c640 1 -- 172.21.15.69:0/1000632713 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51680d4570 msgr2=0x7f51680cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:12.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.118+0000 7f516cc7c640 1 --2- 172.21.15.69:0/1000632713 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51680d4570 0x7f51680cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:12.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.118+0000 7f516cc7c640 1 -- 172.21.15.69:0/1000632713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51680b2dc0 msgr2=0x7f51681f9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:12.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.118+0000 7f516cc7c640 1 --2- 172.21.15.69:0/1000632713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51680b2dc0 0x7f51681f9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:12.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.118+0000 7f516cc7c640 1 -- 172.21.15.69:0/1000632713 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51681f6d20 con 0x7f516806e8f0 2024-04-25T11:00:12.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.118+0000 7f516d47d640 1 --2- 172.21.15.69:0/1000632713 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51680d4570 0x7f51680cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:00:12.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.118+0000 7f5167fff640 1 --2- 172.21.15.69:0/1000632713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51680b2dc0 0x7f51681f9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:00:12.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.118+0000 7f516cc7c640 1 --2- 172.21.15.69:0/1000632713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f516806e8f0 0x7f51681f67e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5158034040 tx=0x7f51580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:12.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.119+0000 7f51657fa640 1 -- 172.21.15.69:0/1000632713 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5158041030 con 0x7f516806e8f0 2024-04-25T11:00:12.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.119+0000 7f51657fa640 1 -- 172.21.15.69:0/1000632713 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5158003c50 con 0x7f516806e8f0 2024-04-25T11:00:12.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.119+0000 7f516eee0640 1 -- 172.21.15.69:0/1000632713 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51680b8700 con 0x7f516806e8f0 2024-04-25T11:00:12.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.119+0000 7f51657fa640 1 -- 172.21.15.69:0/1000632713 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5158004030 con 0x7f516806e8f0 2024-04-25T11:00:12.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.119+0000 7f516eee0640 1 -- 172.21.15.69:0/1000632713 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51681f7510 con 0x7f516806e8f0 2024-04-25T11:00:12.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.119+0000 7f516eee0640 1 -- 172.21.15.69:0/1000632713 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51681f7750 con 0x7f516806e8f0 2024-04-25T11:00:12.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.120+0000 7f51657fa640 1 -- 172.21.15.69:0/1000632713 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5158044070 con 0x7f516806e8f0 2024-04-25T11:00:12.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.121+0000 7f51657fa640 1 --2- 172.21.15.69:0/1000632713 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f51540744b0 0x7f51540769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:12.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.121+0000 7f5167fff640 1 --2- 172.21.15.69:0/1000632713 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f51540744b0 0x7f51540769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:12.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.121+0000 7f51657fa640 1 -- 172.21.15.69:0/1000632713 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f51580b5dd0 con 0x7f516806e8f0 2024-04-25T11:00:12.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.121+0000 7f5167fff640 1 --2- 172.21.15.69:0/1000632713 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f51540744b0 0x7f51540769c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f51680a0d60 tx=0x7f515c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:12.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.125+0000 7f51657fa640 1 -- 172.21.15.69:0/1000632713 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f51681f7750 con 0x7f516806e8f0 2024-04-25T11:00:12.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.370+0000 7f516eee0640 1 -- 172.21.15.69:0/1000632713 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f5168050740 con 0x7f516806e8f0 2024-04-25T11:00:12.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.371+0000 7f51657fa640 1 -- 172.21.15.69:0/1000632713 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1581 (secure 0 0 0) 0x7f51580b7640 con 0x7f516806e8f0 2024-04-25T11:00:12.372 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:00:12.372 INFO:teuthology.orchestra.run.smithi069.stdout:{"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":78,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":31,"num_osds":8,"num_up_osds":8,"osd_up_since":1714042745,"num_in_osds":8,"osd_in_since":1714042736,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":225042432,"bytes_avail":772869070848,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":15,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":5034}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-04-25T11:00:08.415623+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T11:00:12.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.373+0000 7f516eee0640 1 -- 172.21.15.69:0/1000632713 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f51540744b0 msgr2=0x7f51540769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:12.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.373+0000 7f516eee0640 1 --2- 172.21.15.69:0/1000632713 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f51540744b0 0x7f51540769c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f51680a0d60 tx=0x7f515c008040 comp rx=0 tx=0).stop 2024-04-25T11:00:12.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.373+0000 7f516eee0640 1 -- 172.21.15.69:0/1000632713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f516806e8f0 msgr2=0x7f51681f67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:12.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.373+0000 7f516eee0640 1 --2- 172.21.15.69:0/1000632713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f516806e8f0 0x7f51681f67e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5158034040 tx=0x7f51580046c0 comp rx=0 tx=0).stop 2024-04-25T11:00:12.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.373+0000 7f516eee0640 1 -- 172.21.15.69:0/1000632713 shutdown_connections 2024-04-25T11:00:12.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.373+0000 7f516eee0640 1 --2- 172.21.15.69:0/1000632713 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f51540744b0 0x7f51540769c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:12.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.373+0000 7f516eee0640 1 --2- 172.21.15.69:0/1000632713 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51680d4570 0x7f51680cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:12.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.373+0000 7f516eee0640 1 --2- 172.21.15.69:0/1000632713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51680b2dc0 0x7f51681f9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:12.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.373+0000 7f516eee0640 1 --2- 172.21.15.69:0/1000632713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f516806e8f0 0x7f51681f67e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:12.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.373+0000 7f516eee0640 1 -- 172.21.15.69:0/1000632713 >> 172.21.15.69:0/1000632713 conn(0x7f51680d3a50 msgr2=0x7f51680cf8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:12.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.373+0000 7f516eee0640 1 -- 172.21.15.69:0/1000632713 shutdown_connections 2024-04-25T11:00:12.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.373+0000 7f516eee0640 1 -- 172.21.15.69:0/1000632713 wait complete. 2024-04-25T11:00:12.385 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T11:00:12.385 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:00:12.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.492+0000 7f2d79a8c640 1 Processor -- start 2024-04-25T11:00:12.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.492+0000 7f2d79a8c640 1 -- start start 2024-04-25T11:00:12.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.493+0000 7f2d79a8c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2d740cf770 0x7f2d74162590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:12.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.493+0000 7f2d79a8c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d74162ad0 0x7f2d740b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:12.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.493+0000 7f2d79a8c640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d740cfc30 con 0x7f2d740b91d0 2024-04-25T11:00:12.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.493+0000 7f2d79a8c640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f2d740d1ff0 con 0x7f2d740cf770 2024-04-25T11:00:12.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.493+0000 7f2d79a8c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2d740d2890 con 0x7f2d74162ad0 2024-04-25T11:00:12.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.493+0000 7f2d73fff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d74162ad0 0x7f2d740b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:12.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.493+0000 7f2d72ffd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2d740cf770 0x7f2d74162590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:12.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.493+0000 7f2d73fff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d74162ad0 0x7f2d740b0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.69:60578/0 (socket says 172.21.15.69:60578) 2024-04-25T11:00:12.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.494+0000 7f2d73fff640 1 -- 172.21.15.69:0/2930315684 learned_addr learned my addr 172.21.15.69:0/2930315684 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:12.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.493+0000 7f2d737fe640 1 --1- 172.21.15.69:0/2930315684 >> v1:172.21.15.146:6789/0 conn(0x7f2d740b91d0 0x7f2d74161e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.69:51574/0 (socket says 172.21.15.69:51574) 2024-04-25T11:00:12.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.494+0000 7f2d73fff640 1 -- 172.21.15.69:0/2930315684 >> v2:172.21.15.146:3300/0 conn(0x7f2d740cf770 msgr2=0x7f2d74162590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:12.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.494+0000 7f2d73fff640 1 --2- 172.21.15.69:0/2930315684 >> v2:172.21.15.146:3300/0 conn(0x7f2d740cf770 0x7f2d74162590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:12.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.494+0000 7f2d73fff640 1 -- 172.21.15.69:0/2930315684 >> v1:172.21.15.146:6789/0 conn(0x7f2d740b91d0 legacy=0x7f2d74161e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:12.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.494+0000 7f2d73fff640 1 -- 172.21.15.69:0/2930315684 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d740d2240 con 0x7f2d74162ad0 2024-04-25T11:00:12.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.495+0000 7f2d73fff640 1 --2- 172.21.15.69:0/2930315684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d74162ad0 0x7f2d740b0690 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c009e50 tx=0x7f2d6c02a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=320734baa6116406 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:12.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.495+0000 7f2d727fc640 1 -- 172.21.15.69:0/2930315684 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c036070 con 0x7f2d74162ad0 2024-04-25T11:00:12.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.495+0000 7f2d727fc640 1 -- 172.21.15.69:0/2930315684 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d6c031600 con 0x7f2d74162ad0 2024-04-25T11:00:12.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.495+0000 7f2d727fc640 1 -- 172.21.15.69:0/2930315684 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c0319d0 con 0x7f2d74162ad0 2024-04-25T11:00:12.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.496+0000 7f2d79a8c640 1 -- 172.21.15.69:0/2930315684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d74162ad0 msgr2=0x7f2d740b0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:12.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.496+0000 7f2d79a8c640 1 --2- 172.21.15.69:0/2930315684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d74162ad0 0x7f2d740b0690 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c009e50 tx=0x7f2d6c02a680 comp rx=0 tx=0).stop 2024-04-25T11:00:12.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.496+0000 7f2d79a8c640 1 -- 172.21.15.69:0/2930315684 shutdown_connections 2024-04-25T11:00:12.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.496+0000 7f2d79a8c640 1 --2- 172.21.15.69:0/2930315684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d74162ad0 0x7f2d740b0690 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:12.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.496+0000 7f2d79a8c640 1 --2- 172.21.15.69:0/2930315684 >> v2:172.21.15.146:3300/0 conn(0x7f2d740cf770 0x7f2d74162590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:12.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.496+0000 7f2d79a8c640 1 -- 172.21.15.69:0/2930315684 >> 172.21.15.69:0/2930315684 conn(0x7f2d740d3aa0 msgr2=0x7f2d740d3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:12.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.496+0000 7f2d79a8c640 1 -- 172.21.15.69:0/2930315684 shutdown_connections 2024-04-25T11:00:12.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.496+0000 7f2d79a8c640 1 -- 172.21.15.69:0/2930315684 wait complete. 2024-04-25T11:00:12.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.497+0000 7f2d79a8c640 1 Processor -- start 2024-04-25T11:00:12.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.498+0000 7f2d79a8c640 1 -- start start 2024-04-25T11:00:12.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.499+0000 7f2d79a8c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d740b91d0 0x7f2d740b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:12.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.499+0000 7f2d79a8c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d74162ad0 0x7f2d740c6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:12.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.499+0000 7f2d79a8c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d74154be0 0x7f2d74154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:12.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.499+0000 7f2d79a8c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2d740b24a0 con 0x7f2d740b91d0 2024-04-25T11:00:12.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.499+0000 7f2d79a8c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2d740b2320 con 0x7f2d74162ad0 2024-04-25T11:00:12.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.499+0000 7f2d79a8c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2d740b0bd0 con 0x7f2d74154be0 2024-04-25T11:00:12.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.499+0000 7f2d737fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d740b91d0 0x7f2d740b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:12.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.499+0000 7f2d737fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d740b91d0 0x7f2d740b26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36624/0 (socket says 172.21.15.69:36624) 2024-04-25T11:00:12.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.499+0000 7f2d737fe640 1 -- 172.21.15.69:0/4216594164 learned_addr learned my addr 172.21.15.69:0/4216594164 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:12.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.499+0000 7f2d73fff640 1 --2- 172.21.15.69:0/4216594164 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d74154be0 0x7f2d74154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:12.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.499+0000 7f2d737fe640 1 -- 172.21.15.69:0/4216594164 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d74154be0 msgr2=0x7f2d74154ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:12.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.499+0000 7f2d72ffd640 1 --2- 172.21.15.69:0/4216594164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d74162ad0 0x7f2d740c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:12.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.499+0000 7f2d737fe640 1 --2- 172.21.15.69:0/4216594164 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d74154be0 0x7f2d74154ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:12.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.499+0000 7f2d737fe640 1 -- 172.21.15.69:0/4216594164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d74162ad0 msgr2=0x7f2d740c6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:12.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.499+0000 7f2d737fe640 1 --2- 172.21.15.69:0/4216594164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d74162ad0 0x7f2d740c6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:12.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.499+0000 7f2d737fe640 1 -- 172.21.15.69:0/4216594164 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d740b2bf0 con 0x7f2d740b91d0 2024-04-25T11:00:12.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.500+0000 7f2d737fe640 1 --2- 172.21.15.69:0/4216594164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d740b91d0 0x7f2d740b26b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400c3f0 tx=0x7f2d6400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:12.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.500+0000 7f2d5bfff640 1 -- 172.21.15.69:0/4216594164 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d640074e0 con 0x7f2d740b91d0 2024-04-25T11:00:12.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.500+0000 7f2d5bfff640 1 -- 172.21.15.69:0/4216594164 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d64007680 con 0x7f2d740b91d0 2024-04-25T11:00:12.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.500+0000 7f2d5bfff640 1 -- 172.21.15.69:0/4216594164 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d64004a00 con 0x7f2d740b91d0 2024-04-25T11:00:12.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.500+0000 7f2d79a8c640 1 -- 172.21.15.69:0/4216594164 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d7406e180 con 0x7f2d740b91d0 2024-04-25T11:00:12.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.500+0000 7f2d79a8c640 1 -- 172.21.15.69:0/4216594164 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d7406e740 con 0x7f2d740b91d0 2024-04-25T11:00:12.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.501+0000 7f2d79a8c640 1 -- 172.21.15.69:0/4216594164 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d74050740 con 0x7f2d740b91d0 2024-04-25T11:00:12.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.501+0000 7f2d5bfff640 1 -- 172.21.15.69:0/4216594164 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2d6401a020 con 0x7f2d740b91d0 2024-04-25T11:00:12.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.502+0000 7f2d5bfff640 1 --2- 172.21.15.69:0/4216594164 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f2d600744e0 0x7f2d600769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:12.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.502+0000 7f2d5bfff640 1 -- 172.21.15.69:0/4216594164 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f2d640948b0 con 0x7f2d740b91d0 2024-04-25T11:00:12.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.502+0000 7f2d72ffd640 1 --2- 172.21.15.69:0/4216594164 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f2d600744e0 0x7f2d600769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:12.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.502+0000 7f2d72ffd640 1 --2- 172.21.15.69:0/4216594164 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f2d600744e0 0x7f2d600769f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2d74155db0 tx=0x7f2d68008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:12.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.506+0000 7f2d5bfff640 1 -- 172.21.15.69:0/4216594164 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2d6405f160 con 0x7f2d740b91d0 2024-04-25T11:00:12.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.676+0000 7f2d79a8c640 1 -- 172.21.15.69:0/4216594164 --> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2d740d4700 con 0x7f2d600744e0 2024-04-25T11:00:12.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.682+0000 7f2d5bfff640 1 -- 172.21.15.69:0/4216594164 <== mgr.4107 v2:172.21.15.69:6832/1107421267 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251944 (secure 0 0 0) 0x7f2d740d4700 con 0x7f2d600744e0 2024-04-25T11:00:12.685 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:00:12.687 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-04-25T11:00:12.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.685+0000 7f2d79a8c640 1 -- 172.21.15.69:0/4216594164 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f2d600744e0 msgr2=0x7f2d600769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:12.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.685+0000 7f2d79a8c640 1 --2- 172.21.15.69:0/4216594164 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f2d600744e0 0x7f2d600769f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2d74155db0 tx=0x7f2d68008040 comp rx=0 tx=0).stop 2024-04-25T11:00:12.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.685+0000 7f2d79a8c640 1 -- 172.21.15.69:0/4216594164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d740b91d0 msgr2=0x7f2d740b26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:12.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.685+0000 7f2d79a8c640 1 --2- 172.21.15.69:0/4216594164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d740b91d0 0x7f2d740b26b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400c3f0 tx=0x7f2d6400c8b0 comp rx=0 tx=0).stop 2024-04-25T11:00:12.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.685+0000 7f2d79a8c640 1 -- 172.21.15.69:0/4216594164 shutdown_connections 2024-04-25T11:00:12.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.685+0000 7f2d79a8c640 1 --2- 172.21.15.69:0/4216594164 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f2d600744e0 0x7f2d600769f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:12.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.685+0000 7f2d79a8c640 1 --2- 172.21.15.69:0/4216594164 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d74154be0 0x7f2d74154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:12.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.685+0000 7f2d79a8c640 1 --2- 172.21.15.69:0/4216594164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d74162ad0 0x7f2d740c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:12.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.685+0000 7f2d79a8c640 1 --2- 172.21.15.69:0/4216594164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d740b91d0 0x7f2d740b26b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:12.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.685+0000 7f2d79a8c640 1 -- 172.21.15.69:0/4216594164 >> 172.21.15.69:0/4216594164 conn(0x7f2d740d3aa0 msgr2=0x7f2d74161de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:12.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.686+0000 7f2d79a8c640 1 -- 172.21.15.69:0/4216594164 shutdown_connections 2024-04-25T11:00:12.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:12.686+0000 7f2d79a8c640 1 -- 172.21.15.69:0/4216594164 wait complete. 2024-04-25T11:00:12.697 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2024-04-25T11:00:12.416305+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":177,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220148,"kb_used_data":3944,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754572,"statfs":{"total":773094113280,"available":772868681728,"internally_reserved":0,"allocated":4038656,"data_stored":2261156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.174560"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265179+0000","last_change":"2024-04-25T11:00:07.262796+0000","last_active":"2024-04-25T11:00:08.265179+0000","last_peered":"2024-04-25T11:00:08.265179+0000","last_clean":"2024-04-25T11:00:08.265179+0000","last_became_active":"2024-04-25T11:00:07.262396+0000","last_became_peered":"2024-04-25T11:00:07.262396+0000","last_unstale":"2024-04-25T11:00:08.265179+0000","last_undegraded":"2024-04-25T11:00:08.265179+0000","last_fullsized":"2024-04-25T11:00:08.265179+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:46.074685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835725+0000","last_change":"2024-04-25T11:00:07.258056+0000","last_active":"2024-04-25T11:00:08.835725+0000","last_peered":"2024-04-25T11:00:08.835725+0000","last_clean":"2024-04-25T11:00:08.835725+0000","last_became_active":"2024-04-25T11:00:07.257741+0000","last_became_peered":"2024-04-25T11:00:07.257741+0000","last_unstale":"2024-04-25T11:00:08.835725+0000","last_undegraded":"2024-04-25T11:00:08.835725+0000","last_fullsized":"2024-04-25T11:00:08.835725+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:30.196237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.334119+0000","last_change":"2024-04-25T11:00:07.270241+0000","last_active":"2024-04-25T11:00:08.334119+0000","last_peered":"2024-04-25T11:00:08.334119+0000","last_clean":"2024-04-25T11:00:08.334119+0000","last_became_active":"2024-04-25T11:00:07.269930+0000","last_became_peered":"2024-04-25T11:00:07.269930+0000","last_unstale":"2024-04-25T11:00:08.334119+0000","last_undegraded":"2024-04-25T11:00:08.334119+0000","last_fullsized":"2024-04-25T11:00:08.334119+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:03.486987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.259807+0000","last_change":"2024-04-25T11:00:07.261375+0000","last_active":"2024-04-25T11:00:08.259807+0000","last_peered":"2024-04-25T11:00:08.259807+0000","last_clean":"2024-04-25T11:00:08.259807+0000","last_became_active":"2024-04-25T11:00:07.261092+0000","last_became_peered":"2024-04-25T11:00:07.261092+0000","last_unstale":"2024-04-25T11:00:08.259807+0000","last_undegraded":"2024-04-25T11:00:08.259807+0000","last_fullsized":"2024-04-25T11:00:08.259807+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:20.203175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.834594+0000","last_change":"2024-04-25T11:00:07.266720+0000","last_active":"2024-04-25T11:00:08.834594+0000","last_peered":"2024-04-25T11:00:08.834594+0000","last_clean":"2024-04-25T11:00:08.834594+0000","last_became_active":"2024-04-25T11:00:07.266441+0000","last_became_peered":"2024-04-25T11:00:07.266441+0000","last_unstale":"2024-04-25T11:00:08.834594+0000","last_undegraded":"2024-04-25T11:00:08.834594+0000","last_fullsized":"2024-04-25T11:00:08.834594+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:56.938301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835056+0000","last_change":"2024-04-25T11:00:07.266161+0000","last_active":"2024-04-25T11:00:08.835056+0000","last_peered":"2024-04-25T11:00:08.835056+0000","last_clean":"2024-04-25T11:00:08.835056+0000","last_became_active":"2024-04-25T11:00:07.265871+0000","last_became_peered":"2024-04-25T11:00:07.265871+0000","last_unstale":"2024-04-25T11:00:08.835056+0000","last_undegraded":"2024-04-25T11:00:08.835056+0000","last_fullsized":"2024-04-25T11:00:08.835056+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:40.917365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835236+0000","last_change":"2024-04-25T11:00:07.266379+0000","last_active":"2024-04-25T11:00:08.835236+0000","last_peered":"2024-04-25T11:00:08.835236+0000","last_clean":"2024-04-25T11:00:08.835236+0000","last_became_active":"2024-04-25T11:00:07.266022+0000","last_became_peered":"2024-04-25T11:00:07.266022+0000","last_unstale":"2024-04-25T11:00:08.835236+0000","last_undegraded":"2024-04-25T11:00:08.835236+0000","last_fullsized":"2024-04-25T11:00:08.835236+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:30.196237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.834900+0000","last_change":"2024-04-25T11:00:07.264408+0000","last_active":"2024-04-25T11:00:08.834900+0000","last_peered":"2024-04-25T11:00:08.834900+0000","last_clean":"2024-04-25T11:00:08.834900+0000","last_became_active":"2024-04-25T11:00:07.264015+0000","last_became_peered":"2024-04-25T11:00:07.264015+0000","last_unstale":"2024-04-25T11:00:08.834900+0000","last_undegraded":"2024-04-25T11:00:08.834900+0000","last_fullsized":"2024-04-25T11:00:08.834900+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:40.917365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265879+0000","last_change":"2024-04-25T11:00:07.270707+0000","last_active":"2024-04-25T11:00:08.265879+0000","last_peered":"2024-04-25T11:00:08.265879+0000","last_clean":"2024-04-25T11:00:08.265879+0000","last_became_active":"2024-04-25T11:00:07.267820+0000","last_became_peered":"2024-04-25T11:00:07.267820+0000","last_unstale":"2024-04-25T11:00:08.265879+0000","last_undegraded":"2024-04-25T11:00:08.265879+0000","last_fullsized":"2024-04-25T11:00:08.265879+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:21.078922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.260109+0000","last_change":"2024-04-25T11:00:07.271255+0000","last_active":"2024-04-25T11:00:08.260109+0000","last_peered":"2024-04-25T11:00:08.260109+0000","last_clean":"2024-04-25T11:00:08.260109+0000","last_became_active":"2024-04-25T11:00:07.270847+0000","last_became_peered":"2024-04-25T11:00:07.270847+0000","last_unstale":"2024-04-25T11:00:08.260109+0000","last_undegraded":"2024-04-25T11:00:08.260109+0000","last_fullsized":"2024-04-25T11:00:08.260109+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:30.194502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835431+0000","last_change":"2024-04-25T11:00:07.270525+0000","last_active":"2024-04-25T11:00:08.835431+0000","last_peered":"2024-04-25T11:00:08.835431+0000","last_clean":"2024-04-25T11:00:08.835431+0000","last_became_active":"2024-04-25T11:00:07.270208+0000","last_became_peered":"2024-04-25T11:00:07.270208+0000","last_unstale":"2024-04-25T11:00:08.835431+0000","last_undegraded":"2024-04-25T11:00:08.835431+0000","last_fullsized":"2024-04-25T11:00:08.835431+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:07.367840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.334090+0000","last_change":"2024-04-25T11:00:07.265019+0000","last_active":"2024-04-25T11:00:08.334090+0000","last_peered":"2024-04-25T11:00:08.334090+0000","last_clean":"2024-04-25T11:00:08.334090+0000","last_became_active":"2024-04-25T11:00:07.264666+0000","last_became_peered":"2024-04-25T11:00:07.264666+0000","last_unstale":"2024-04-25T11:00:08.334090+0000","last_undegraded":"2024-04-25T11:00:08.334090+0000","last_fullsized":"2024-04-25T11:00:08.334090+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:51.247079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.834823+0000","last_change":"2024-04-25T11:00:07.267683+0000","last_active":"2024-04-25T11:00:08.834823+0000","last_peered":"2024-04-25T11:00:08.834823+0000","last_clean":"2024-04-25T11:00:08.834823+0000","last_became_active":"2024-04-25T11:00:07.267400+0000","last_became_peered":"2024-04-25T11:00:07.267400+0000","last_unstale":"2024-04-25T11:00:08.834823+0000","last_undegraded":"2024-04-25T11:00:08.834823+0000","last_fullsized":"2024-04-25T11:00:08.834823+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:48.605487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835217+0000","last_change":"2024-04-25T11:00:07.265397+0000","last_active":"2024-04-25T11:00:08.835217+0000","last_peered":"2024-04-25T11:00:08.835217+0000","last_clean":"2024-04-25T11:00:08.835217+0000","last_became_active":"2024-04-25T11:00:07.264141+0000","last_became_peered":"2024-04-25T11:00:07.264141+0000","last_unstale":"2024-04-25T11:00:08.835217+0000","last_undegraded":"2024-04-25T11:00:08.835217+0000","last_fullsized":"2024-04-25T11:00:08.835217+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:51.247079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.334023+0000","last_change":"2024-04-25T11:00:07.266066+0000","last_active":"2024-04-25T11:00:08.334023+0000","last_peered":"2024-04-25T11:00:08.334023+0000","last_clean":"2024-04-25T11:00:08.334023+0000","last_became_active":"2024-04-25T11:00:07.264905+0000","last_became_peered":"2024-04-25T11:00:07.264905+0000","last_unstale":"2024-04-25T11:00:08.334023+0000","last_undegraded":"2024-04-25T11:00:08.334023+0000","last_fullsized":"2024-04-25T11:00:08.334023+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:40.917365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835627+0000","last_change":"2024-04-25T11:00:07.273989+0000","last_active":"2024-04-25T11:00:08.835627+0000","last_peered":"2024-04-25T11:00:08.835627+0000","last_clean":"2024-04-25T11:00:08.835627+0000","last_became_active":"2024-04-25T11:00:07.273658+0000","last_became_peered":"2024-04-25T11:00:07.273658+0000","last_unstale":"2024-04-25T11:00:08.835627+0000","last_undegraded":"2024-04-25T11:00:08.835627+0000","last_fullsized":"2024-04-25T11:00:08.835627+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:10.131991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835306+0000","last_change":"2024-04-25T11:00:07.267254+0000","last_active":"2024-04-25T11:00:08.835306+0000","last_peered":"2024-04-25T11:00:08.835306+0000","last_clean":"2024-04-25T11:00:08.835306+0000","last_became_active":"2024-04-25T11:00:07.266905+0000","last_became_peered":"2024-04-25T11:00:07.266905+0000","last_unstale":"2024-04-25T11:00:08.835306+0000","last_undegraded":"2024-04-25T11:00:08.835306+0000","last_fullsized":"2024-04-25T11:00:08.835306+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:09.261812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835790+0000","last_change":"2024-04-25T11:00:07.267161+0000","last_active":"2024-04-25T11:00:08.835790+0000","last_peered":"2024-04-25T11:00:08.835790+0000","last_clean":"2024-04-25T11:00:08.835790+0000","last_became_active":"2024-04-25T11:00:07.266877+0000","last_became_peered":"2024-04-25T11:00:07.266877+0000","last_unstale":"2024-04-25T11:00:08.835790+0000","last_undegraded":"2024-04-25T11:00:08.835790+0000","last_fullsized":"2024-04-25T11:00:08.835790+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:09.261812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.834850+0000","last_change":"2024-04-25T11:00:07.273250+0000","last_active":"2024-04-25T11:00:08.834850+0000","last_peered":"2024-04-25T11:00:08.834850+0000","last_clean":"2024-04-25T11:00:08.834850+0000","last_became_active":"2024-04-25T11:00:07.272949+0000","last_became_peered":"2024-04-25T11:00:07.272949+0000","last_unstale":"2024-04-25T11:00:08.834850+0000","last_undegraded":"2024-04-25T11:00:08.834850+0000","last_fullsized":"2024-04-25T11:00:08.834850+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:38.615241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835058+0000","last_change":"2024-04-25T11:00:07.261730+0000","last_active":"2024-04-25T11:00:08.835058+0000","last_peered":"2024-04-25T11:00:08.835058+0000","last_clean":"2024-04-25T11:00:08.835058+0000","last_became_active":"2024-04-25T11:00:07.261415+0000","last_became_peered":"2024-04-25T11:00:07.261415+0000","last_unstale":"2024-04-25T11:00:08.835058+0000","last_undegraded":"2024-04-25T11:00:08.835058+0000","last_fullsized":"2024-04-25T11:00:08.835058+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:28.820953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265693+0000","last_change":"2024-04-25T11:00:07.258119+0000","last_active":"2024-04-25T11:00:08.265693+0000","last_peered":"2024-04-25T11:00:08.265693+0000","last_clean":"2024-04-25T11:00:08.265693+0000","last_became_active":"2024-04-25T11:00:07.257207+0000","last_became_peered":"2024-04-25T11:00:07.257207+0000","last_unstale":"2024-04-25T11:00:08.265693+0000","last_undegraded":"2024-04-25T11:00:08.265693+0000","last_fullsized":"2024-04-25T11:00:08.265693+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:50.570436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265357+0000","last_change":"2024-04-25T11:00:07.259122+0000","last_active":"2024-04-25T11:00:08.265357+0000","last_peered":"2024-04-25T11:00:08.265357+0000","last_clean":"2024-04-25T11:00:08.265357+0000","last_became_active":"2024-04-25T11:00:07.258787+0000","last_became_peered":"2024-04-25T11:00:07.258787+0000","last_unstale":"2024-04-25T11:00:08.265357+0000","last_undegraded":"2024-04-25T11:00:08.265357+0000","last_fullsized":"2024-04-25T11:00:08.265357+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:56.469604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.256876+0000","last_change":"2024-04-25T11:00:07.259818+0000","last_active":"2024-04-25T11:00:08.256876+0000","last_peered":"2024-04-25T11:00:08.256876+0000","last_clean":"2024-04-25T11:00:08.256876+0000","last_became_active":"2024-04-25T11:00:07.259187+0000","last_became_peered":"2024-04-25T11:00:07.259187+0000","last_unstale":"2024-04-25T11:00:08.256876+0000","last_undegraded":"2024-04-25T11:00:08.256876+0000","last_fullsized":"2024-04-25T11:00:08.256876+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:17.734316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.334087+0000","last_change":"2024-04-25T11:00:07.267648+0000","last_active":"2024-04-25T11:00:08.334087+0000","last_peered":"2024-04-25T11:00:08.334087+0000","last_clean":"2024-04-25T11:00:08.334087+0000","last_became_active":"2024-04-25T11:00:07.267349+0000","last_became_peered":"2024-04-25T11:00:07.267349+0000","last_unstale":"2024-04-25T11:00:08.334087+0000","last_undegraded":"2024-04-25T11:00:08.334087+0000","last_fullsized":"2024-04-25T11:00:08.334087+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:30.196237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.260262+0000","last_change":"2024-04-25T11:00:07.266306+0000","last_active":"2024-04-25T11:00:08.260262+0000","last_peered":"2024-04-25T11:00:08.260262+0000","last_clean":"2024-04-25T11:00:08.260262+0000","last_became_active":"2024-04-25T11:00:07.265935+0000","last_became_peered":"2024-04-25T11:00:07.265935+0000","last_unstale":"2024-04-25T11:00:08.260262+0000","last_undegraded":"2024-04-25T11:00:08.260262+0000","last_fullsized":"2024-04-25T11:00:08.260262+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:04.418320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835817+0000","last_change":"2024-04-25T11:00:07.262516+0000","last_active":"2024-04-25T11:00:08.835817+0000","last_peered":"2024-04-25T11:00:08.835817+0000","last_clean":"2024-04-25T11:00:08.835817+0000","last_became_active":"2024-04-25T11:00:07.261840+0000","last_became_peered":"2024-04-25T11:00:07.261840+0000","last_unstale":"2024-04-25T11:00:08.835817+0000","last_undegraded":"2024-04-25T11:00:08.835817+0000","last_fullsized":"2024-04-25T11:00:08.835817+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:19.428784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.260256+0000","last_change":"2024-04-25T11:00:07.265227+0000","last_active":"2024-04-25T11:00:08.260256+0000","last_peered":"2024-04-25T11:00:08.260256+0000","last_clean":"2024-04-25T11:00:08.260256+0000","last_became_active":"2024-04-25T11:00:07.264946+0000","last_became_peered":"2024-04-25T11:00:07.264946+0000","last_unstale":"2024-04-25T11:00:08.260256+0000","last_undegraded":"2024-04-25T11:00:08.260256+0000","last_fullsized":"2024-04-25T11:00:08.260256+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:21.078922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.334339+0000","last_change":"2024-04-25T11:00:07.259601+0000","last_active":"2024-04-25T11:00:08.334339+0000","last_peered":"2024-04-25T11:00:08.334339+0000","last_clean":"2024-04-25T11:00:08.334339+0000","last_became_active":"2024-04-25T11:00:07.258529+0000","last_became_peered":"2024-04-25T11:00:07.258529+0000","last_unstale":"2024-04-25T11:00:08.334339+0000","last_undegraded":"2024-04-25T11:00:08.334339+0000","last_fullsized":"2024-04-25T11:00:08.334339+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:09.261812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265850+0000","last_change":"2024-04-25T11:00:07.266448+0000","last_active":"2024-04-25T11:00:08.265850+0000","last_peered":"2024-04-25T11:00:08.265850+0000","last_clean":"2024-04-25T11:00:08.265850+0000","last_became_active":"2024-04-25T11:00:07.266118+0000","last_became_peered":"2024-04-25T11:00:07.266118+0000","last_unstale":"2024-04-25T11:00:08.265850+0000","last_undegraded":"2024-04-25T11:00:08.265850+0000","last_fullsized":"2024-04-25T11:00:08.265850+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:30.194502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835040+0000","last_change":"2024-04-25T11:00:07.266442+0000","last_active":"2024-04-25T11:00:08.835040+0000","last_peered":"2024-04-25T11:00:08.835040+0000","last_clean":"2024-04-25T11:00:08.835040+0000","last_became_active":"2024-04-25T11:00:07.266101+0000","last_became_peered":"2024-04-25T11:00:07.266101+0000","last_unstale":"2024-04-25T11:00:08.835040+0000","last_undegraded":"2024-04-25T11:00:08.835040+0000","last_fullsized":"2024-04-25T11:00:08.835040+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:03.486987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.264808+0000","last_change":"2024-04-25T11:00:07.260993+0000","last_active":"2024-04-25T11:00:08.264808+0000","last_peered":"2024-04-25T11:00:08.264808+0000","last_clean":"2024-04-25T11:00:08.264808+0000","last_became_active":"2024-04-25T11:00:07.260408+0000","last_became_peered":"2024-04-25T11:00:07.260408+0000","last_unstale":"2024-04-25T11:00:08.264808+0000","last_undegraded":"2024-04-25T11:00:08.264808+0000","last_fullsized":"2024-04-25T11:00:08.264808+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:24.519419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.834885+0000","last_change":"2024-04-25T11:00:07.272840+0000","last_active":"2024-04-25T11:00:08.834885+0000","last_peered":"2024-04-25T11:00:08.834885+0000","last_clean":"2024-04-25T11:00:08.834885+0000","last_became_active":"2024-04-25T11:00:07.272454+0000","last_became_peered":"2024-04-25T11:00:07.272454+0000","last_unstale":"2024-04-25T11:00:08.834885+0000","last_undegraded":"2024-04-25T11:00:08.834885+0000","last_fullsized":"2024-04-25T11:00:08.834885+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:38.615241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.257217+0000","last_change":"2024-04-25T11:00:07.258519+0000","last_active":"2024-04-25T11:00:08.257217+0000","last_peered":"2024-04-25T11:00:08.257217+0000","last_clean":"2024-04-25T11:00:08.257217+0000","last_became_active":"2024-04-25T11:00:07.258222+0000","last_became_peered":"2024-04-25T11:00:07.258222+0000","last_unstale":"2024-04-25T11:00:08.257217+0000","last_undegraded":"2024-04-25T11:00:08.257217+0000","last_fullsized":"2024-04-25T11:00:08.257217+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:56.469604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"30'4","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265498+0000","last_change":"2024-04-25T11:00:06.257849+0000","last_active":"2024-04-25T11:00:08.265498+0000","last_peered":"2024-04-25T11:00:08.265498+0000","last_clean":"2024-04-25T11:00:08.265498+0000","last_became_active":"2024-04-25T11:00:06.257452+0000","last_became_peered":"2024-04-25T11:00:06.257452+0000","last_unstale":"2024-04-25T11:00:08.265498+0000","last_undegraded":"2024-04-25T11:00:08.265498+0000","last_fullsized":"2024-04-25T11:00:08.265498+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:37.048525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265371+0000","last_change":"2024-04-25T11:00:07.258826+0000","last_active":"2024-04-25T11:00:08.265371+0000","last_peered":"2024-04-25T11:00:08.265371+0000","last_clean":"2024-04-25T11:00:08.265371+0000","last_became_active":"2024-04-25T11:00:07.258379+0000","last_became_peered":"2024-04-25T11:00:07.258379+0000","last_unstale":"2024-04-25T11:00:08.265371+0000","last_undegraded":"2024-04-25T11:00:08.265371+0000","last_fullsized":"2024-04-25T11:00:08.265371+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:21.042782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.257184+0000","last_change":"2024-04-25T11:00:06.265699+0000","last_active":"2024-04-25T11:00:08.257184+0000","last_peered":"2024-04-25T11:00:08.257184+0000","last_clean":"2024-04-25T11:00:08.257184+0000","last_became_active":"2024-04-25T11:00:06.265112+0000","last_became_peered":"2024-04-25T11:00:06.265112+0000","last_unstale":"2024-04-25T11:00:08.257184+0000","last_undegraded":"2024-04-25T11:00:08.257184+0000","last_fullsized":"2024-04-25T11:00:08.257184+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:02.917078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265501+0000","last_change":"2024-04-25T11:00:07.265206+0000","last_active":"2024-04-25T11:00:08.265501+0000","last_peered":"2024-04-25T11:00:08.265501+0000","last_clean":"2024-04-25T11:00:08.265501+0000","last_became_active":"2024-04-25T11:00:07.264877+0000","last_became_peered":"2024-04-25T11:00:07.264877+0000","last_unstale":"2024-04-25T11:00:08.265501+0000","last_undegraded":"2024-04-25T11:00:08.265501+0000","last_fullsized":"2024-04-25T11:00:08.265501+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:20.203175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.834866+0000","last_change":"2024-04-25T11:00:06.262272+0000","last_active":"2024-04-25T11:00:08.834866+0000","last_peered":"2024-04-25T11:00:08.834866+0000","last_clean":"2024-04-25T11:00:08.834866+0000","last_became_active":"2024-04-25T11:00:06.261982+0000","last_became_peered":"2024-04-25T11:00:06.261982+0000","last_unstale":"2024-04-25T11:00:08.834866+0000","last_undegraded":"2024-04-25T11:00:08.834866+0000","last_fullsized":"2024-04-25T11:00:08.834866+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:47.597884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.260431+0000","last_change":"2024-04-25T11:00:07.272099+0000","last_active":"2024-04-25T11:00:08.260431+0000","last_peered":"2024-04-25T11:00:08.260431+0000","last_clean":"2024-04-25T11:00:08.260431+0000","last_became_active":"2024-04-25T11:00:07.271755+0000","last_became_peered":"2024-04-25T11:00:07.271755+0000","last_unstale":"2024-04-25T11:00:08.260431+0000","last_undegraded":"2024-04-25T11:00:08.260431+0000","last_fullsized":"2024-04-25T11:00:08.260431+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:11.896031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835040+0000","last_change":"2024-04-25T11:00:06.257693+0000","last_active":"2024-04-25T11:00:08.835040+0000","last_peered":"2024-04-25T11:00:08.835040+0000","last_clean":"2024-04-25T11:00:08.835040+0000","last_became_active":"2024-04-25T11:00:06.257402+0000","last_became_peered":"2024-04-25T11:00:06.257402+0000","last_unstale":"2024-04-25T11:00:08.835040+0000","last_undegraded":"2024-04-25T11:00:08.835040+0000","last_fullsized":"2024-04-25T11:00:08.835040+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:31.946740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835381+0000","last_change":"2024-04-25T11:00:07.270211+0000","last_active":"2024-04-25T11:00:08.835381+0000","last_peered":"2024-04-25T11:00:08.835381+0000","last_clean":"2024-04-25T11:00:08.835381+0000","last_became_active":"2024-04-25T11:00:07.269871+0000","last_became_peered":"2024-04-25T11:00:07.269871+0000","last_unstale":"2024-04-25T11:00:08.835381+0000","last_undegraded":"2024-04-25T11:00:08.835381+0000","last_fullsized":"2024-04-25T11:00:08.835381+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:02.017251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265828+0000","last_change":"2024-04-25T11:00:06.260758+0000","last_active":"2024-04-25T11:00:08.265828+0000","last_peered":"2024-04-25T11:00:08.265828+0000","last_clean":"2024-04-25T11:00:08.265828+0000","last_became_active":"2024-04-25T11:00:06.260455+0000","last_became_peered":"2024-04-25T11:00:06.260455+0000","last_unstale":"2024-04-25T11:00:08.265828+0000","last_undegraded":"2024-04-25T11:00:08.265828+0000","last_fullsized":"2024-04-25T11:00:08.265828+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:37.048525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.333932+0000","last_change":"2024-04-25T11:00:07.259280+0000","last_active":"2024-04-25T11:00:08.333932+0000","last_peered":"2024-04-25T11:00:08.333932+0000","last_clean":"2024-04-25T11:00:08.333932+0000","last_became_active":"2024-04-25T11:00:07.258640+0000","last_became_peered":"2024-04-25T11:00:07.258640+0000","last_unstale":"2024-04-25T11:00:08.333932+0000","last_undegraded":"2024-04-25T11:00:08.333932+0000","last_fullsized":"2024-04-25T11:00:08.333932+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:38.615241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265374+0000","last_change":"2024-04-25T11:00:06.255855+0000","last_active":"2024-04-25T11:00:08.265374+0000","last_peered":"2024-04-25T11:00:08.265374+0000","last_clean":"2024-04-25T11:00:08.265374+0000","last_became_active":"2024-04-25T11:00:06.255509+0000","last_became_peered":"2024-04-25T11:00:06.255509+0000","last_unstale":"2024-04-25T11:00:08.265374+0000","last_undegraded":"2024-04-25T11:00:08.265374+0000","last_fullsized":"2024-04-25T11:00:08.265374+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:55.930698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.264719+0000","last_change":"2024-04-25T11:00:07.253754+0000","last_active":"2024-04-25T11:00:08.264719+0000","last_peered":"2024-04-25T11:00:08.264719+0000","last_clean":"2024-04-25T11:00:08.264719+0000","last_became_active":"2024-04-25T11:00:07.253424+0000","last_became_peered":"2024-04-25T11:00:07.253424+0000","last_unstale":"2024-04-25T11:00:08.264719+0000","last_undegraded":"2024-04-25T11:00:08.264719+0000","last_fullsized":"2024-04-25T11:00:08.264719+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:17.734316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.834560+0000","last_change":"2024-04-25T11:00:06.259273+0000","last_active":"2024-04-25T11:00:08.834560+0000","last_peered":"2024-04-25T11:00:08.834560+0000","last_clean":"2024-04-25T11:00:08.834560+0000","last_became_active":"2024-04-25T11:00:06.258923+0000","last_became_peered":"2024-04-25T11:00:06.258923+0000","last_unstale":"2024-04-25T11:00:08.834560+0000","last_undegraded":"2024-04-25T11:00:08.834560+0000","last_fullsized":"2024-04-25T11:00:08.834560+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:20.035179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.259700+0000","last_change":"2024-04-25T11:00:07.266213+0000","last_active":"2024-04-25T11:00:08.259700+0000","last_peered":"2024-04-25T11:00:08.259700+0000","last_clean":"2024-04-25T11:00:08.259700+0000","last_became_active":"2024-04-25T11:00:07.265846+0000","last_became_peered":"2024-04-25T11:00:07.265846+0000","last_unstale":"2024-04-25T11:00:08.259700+0000","last_undegraded":"2024-04-25T11:00:08.259700+0000","last_fullsized":"2024-04-25T11:00:08.259700+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:51.748199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.264850+0000","last_change":"2024-04-25T11:00:06.259650+0000","last_active":"2024-04-25T11:00:08.264850+0000","last_peered":"2024-04-25T11:00:08.264850+0000","last_clean":"2024-04-25T11:00:08.264850+0000","last_became_active":"2024-04-25T11:00:06.259152+0000","last_became_peered":"2024-04-25T11:00:06.259152+0000","last_unstale":"2024-04-25T11:00:08.264850+0000","last_undegraded":"2024-04-25T11:00:08.264850+0000","last_fullsized":"2024-04-25T11:00:08.264850+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:30.099257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.334408+0000","last_change":"2024-04-25T11:00:07.265671+0000","last_active":"2024-04-25T11:00:08.334408+0000","last_peered":"2024-04-25T11:00:08.334408+0000","last_clean":"2024-04-25T11:00:08.334408+0000","last_became_active":"2024-04-25T11:00:07.265322+0000","last_became_peered":"2024-04-25T11:00:07.265322+0000","last_unstale":"2024-04-25T11:00:08.334408+0000","last_undegraded":"2024-04-25T11:00:08.334408+0000","last_fullsized":"2024-04-25T11:00:08.334408+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:02.017251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835351+0000","last_change":"2024-04-25T11:00:06.266165+0000","last_active":"2024-04-25T11:00:08.835351+0000","last_peered":"2024-04-25T11:00:08.835351+0000","last_clean":"2024-04-25T11:00:08.835351+0000","last_became_active":"2024-04-25T11:00:06.265773+0000","last_became_peered":"2024-04-25T11:00:06.265773+0000","last_unstale":"2024-04-25T11:00:08.835351+0000","last_undegraded":"2024-04-25T11:00:08.835351+0000","last_fullsized":"2024-04-25T11:00:08.835351+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:10.861006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835282+0000","last_change":"2024-04-25T11:00:07.272301+0000","last_active":"2024-04-25T11:00:08.835282+0000","last_peered":"2024-04-25T11:00:08.835282+0000","last_clean":"2024-04-25T11:00:08.835282+0000","last_became_active":"2024-04-25T11:00:07.271957+0000","last_became_peered":"2024-04-25T11:00:07.271957+0000","last_unstale":"2024-04-25T11:00:08.835282+0000","last_undegraded":"2024-04-25T11:00:08.835282+0000","last_fullsized":"2024-04-25T11:00:08.835282+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:03.486987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.257266+0000","last_change":"2024-04-25T11:00:06.259862+0000","last_active":"2024-04-25T11:00:08.257266+0000","last_peered":"2024-04-25T11:00:08.257266+0000","last_clean":"2024-04-25T11:00:08.257266+0000","last_became_active":"2024-04-25T11:00:06.259570+0000","last_became_peered":"2024-04-25T11:00:06.259570+0000","last_unstale":"2024-04-25T11:00:08.257266+0000","last_undegraded":"2024-04-25T11:00:08.257266+0000","last_fullsized":"2024-04-25T11:00:08.257266+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:47.597884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835894+0000","last_change":"2024-04-25T11:00:07.272966+0000","last_active":"2024-04-25T11:00:08.835894+0000","last_peered":"2024-04-25T11:00:08.835894+0000","last_clean":"2024-04-25T11:00:08.835894+0000","last_became_active":"2024-04-25T11:00:07.272568+0000","last_became_peered":"2024-04-25T11:00:07.272568+0000","last_unstale":"2024-04-25T11:00:08.835894+0000","last_undegraded":"2024-04-25T11:00:08.835894+0000","last_fullsized":"2024-04-25T11:00:08.835894+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:02.017251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"30'3","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.334266+0000","last_change":"2024-04-25T11:00:06.266271+0000","last_active":"2024-04-25T11:00:08.334266+0000","last_peered":"2024-04-25T11:00:08.334266+0000","last_clean":"2024-04-25T11:00:08.334266+0000","last_became_active":"2024-04-25T11:00:06.265815+0000","last_became_peered":"2024-04-25T11:00:06.265815+0000","last_unstale":"2024-04-25T11:00:08.334266+0000","last_undegraded":"2024-04-25T11:00:08.334266+0000","last_fullsized":"2024-04-25T11:00:08.334266+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:20.035179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.334126+0000","last_change":"2024-04-25T11:00:07.270387+0000","last_active":"2024-04-25T11:00:08.334126+0000","last_peered":"2024-04-25T11:00:08.334126+0000","last_clean":"2024-04-25T11:00:08.334126+0000","last_became_active":"2024-04-25T11:00:07.270095+0000","last_became_peered":"2024-04-25T11:00:07.270095+0000","last_unstale":"2024-04-25T11:00:08.334126+0000","last_undegraded":"2024-04-25T11:00:08.334126+0000","last_fullsized":"2024-04-25T11:00:08.334126+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:16.167953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"30'6","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835675+0000","last_change":"2024-04-25T11:00:06.266846+0000","last_active":"2024-04-25T11:00:08.835675+0000","last_peered":"2024-04-25T11:00:08.835675+0000","last_clean":"2024-04-25T11:00:08.835675+0000","last_became_active":"2024-04-25T11:00:06.266553+0000","last_became_peered":"2024-04-25T11:00:06.266553+0000","last_unstale":"2024-04-25T11:00:08.835675+0000","last_undegraded":"2024-04-25T11:00:08.835675+0000","last_fullsized":"2024-04-25T11:00:08.835675+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:59.507845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.259726+0000","last_change":"2024-04-25T11:00:07.268836+0000","last_active":"2024-04-25T11:00:08.259726+0000","last_peered":"2024-04-25T11:00:08.259726+0000","last_clean":"2024-04-25T11:00:08.259726+0000","last_became_active":"2024-04-25T11:00:07.268518+0000","last_became_peered":"2024-04-25T11:00:07.268518+0000","last_unstale":"2024-04-25T11:00:08.259726+0000","last_undegraded":"2024-04-25T11:00:08.259726+0000","last_fullsized":"2024-04-25T11:00:08.259726+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:56.469604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265928+0000","last_change":"2024-04-25T11:00:06.258144+0000","last_active":"2024-04-25T11:00:08.265928+0000","last_peered":"2024-04-25T11:00:08.265928+0000","last_clean":"2024-04-25T11:00:08.265928+0000","last_became_active":"2024-04-25T11:00:06.257775+0000","last_became_peered":"2024-04-25T11:00:06.257775+0000","last_unstale":"2024-04-25T11:00:08.265928+0000","last_undegraded":"2024-04-25T11:00:08.265928+0000","last_fullsized":"2024-04-25T11:00:08.265928+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:03.333540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835562+0000","last_change":"2024-04-25T11:00:07.270774+0000","last_active":"2024-04-25T11:00:08.835562+0000","last_peered":"2024-04-25T11:00:08.835562+0000","last_clean":"2024-04-25T11:00:08.835562+0000","last_became_active":"2024-04-25T11:00:07.270481+0000","last_became_peered":"2024-04-25T11:00:07.270481+0000","last_unstale":"2024-04-25T11:00:08.835562+0000","last_undegraded":"2024-04-25T11:00:08.835562+0000","last_fullsized":"2024-04-25T11:00:08.835562+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:16.167953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.259786+0000","last_change":"2024-04-25T11:00:06.260717+0000","last_active":"2024-04-25T11:00:08.259786+0000","last_peered":"2024-04-25T11:00:08.259786+0000","last_clean":"2024-04-25T11:00:08.259786+0000","last_became_active":"2024-04-25T11:00:06.260433+0000","last_became_peered":"2024-04-25T11:00:06.260433+0000","last_unstale":"2024-04-25T11:00:08.259786+0000","last_undegraded":"2024-04-25T11:00:08.259786+0000","last_fullsized":"2024-04-25T11:00:08.259786+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:02.917078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835298+0000","last_change":"2024-04-25T11:00:07.271908+0000","last_active":"2024-04-25T11:00:08.835298+0000","last_peered":"2024-04-25T11:00:08.835298+0000","last_clean":"2024-04-25T11:00:08.835298+0000","last_became_active":"2024-04-25T11:00:07.271482+0000","last_became_peered":"2024-04-25T11:00:07.271482+0000","last_unstale":"2024-04-25T11:00:08.835298+0000","last_undegraded":"2024-04-25T11:00:08.835298+0000","last_fullsized":"2024-04-25T11:00:08.835298+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:19.428784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.834707+0000","last_change":"2024-04-25T11:00:06.265403+0000","last_active":"2024-04-25T11:00:08.834707+0000","last_peered":"2024-04-25T11:00:08.834707+0000","last_clean":"2024-04-25T11:00:08.834707+0000","last_became_active":"2024-04-25T11:00:06.265062+0000","last_became_peered":"2024-04-25T11:00:06.265062+0000","last_unstale":"2024-04-25T11:00:08.834707+0000","last_undegraded":"2024-04-25T11:00:08.834707+0000","last_fullsized":"2024-04-25T11:00:08.834707+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:45.067082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265012+0000","last_change":"2024-04-25T11:00:07.260724+0000","last_active":"2024-04-25T11:00:08.265012+0000","last_peered":"2024-04-25T11:00:08.265012+0000","last_clean":"2024-04-25T11:00:08.265012+0000","last_became_active":"2024-04-25T11:00:07.259749+0000","last_became_peered":"2024-04-25T11:00:07.259749+0000","last_unstale":"2024-04-25T11:00:08.265012+0000","last_undegraded":"2024-04-25T11:00:08.265012+0000","last_fullsized":"2024-04-25T11:00:08.265012+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:28.820953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"30'8","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.834506+0000","last_change":"2024-04-25T11:00:06.266837+0000","last_active":"2024-04-25T11:00:08.834506+0000","last_peered":"2024-04-25T11:00:08.834506+0000","last_clean":"2024-04-25T11:00:08.834506+0000","last_became_active":"2024-04-25T11:00:06.266003+0000","last_became_peered":"2024-04-25T11:00:06.266003+0000","last_unstale":"2024-04-25T11:00:08.834506+0000","last_undegraded":"2024-04-25T11:00:08.834506+0000","last_fullsized":"2024-04-25T11:00:08.834506+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:23.511816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.264765+0000","last_change":"2024-04-25T11:00:07.261111+0000","last_active":"2024-04-25T11:00:08.264765+0000","last_peered":"2024-04-25T11:00:08.264765+0000","last_clean":"2024-04-25T11:00:08.264765+0000","last_became_active":"2024-04-25T11:00:07.260768+0000","last_became_peered":"2024-04-25T11:00:07.260768+0000","last_unstale":"2024-04-25T11:00:08.264765+0000","last_undegraded":"2024-04-25T11:00:08.264765+0000","last_fullsized":"2024-04-25T11:00:08.264765+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:00.515448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.260341+0000","last_change":"2024-04-25T11:00:06.262237+0000","last_active":"2024-04-25T11:00:08.260341+0000","last_peered":"2024-04-25T11:00:08.260341+0000","last_clean":"2024-04-25T11:00:08.260341+0000","last_became_active":"2024-04-25T11:00:06.261872+0000","last_became_peered":"2024-04-25T11:00:06.261872+0000","last_unstale":"2024-04-25T11:00:08.260341+0000","last_undegraded":"2024-04-25T11:00:08.260341+0000","last_fullsized":"2024-04-25T11:00:08.260341+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:03.333540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.334370+0000","last_change":"2024-04-25T11:00:07.264556+0000","last_active":"2024-04-25T11:00:08.334370+0000","last_peered":"2024-04-25T11:00:08.334370+0000","last_clean":"2024-04-25T11:00:08.334370+0000","last_became_active":"2024-04-25T11:00:07.263925+0000","last_became_peered":"2024-04-25T11:00:07.263925+0000","last_unstale":"2024-04-25T11:00:08.334370+0000","last_undegraded":"2024-04-25T11:00:08.334370+0000","last_fullsized":"2024-04-25T11:00:08.334370+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:19.428784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.264897+0000","last_change":"2024-04-25T11:00:06.257930+0000","last_active":"2024-04-25T11:00:08.264897+0000","last_peered":"2024-04-25T11:00:08.264897+0000","last_clean":"2024-04-25T11:00:08.264897+0000","last_became_active":"2024-04-25T11:00:06.257326+0000","last_became_peered":"2024-04-25T11:00:06.257326+0000","last_unstale":"2024-04-25T11:00:08.264897+0000","last_undegraded":"2024-04-25T11:00:08.264897+0000","last_fullsized":"2024-04-25T11:00:08.264897+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:11.002336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.259925+0000","last_change":"2024-04-25T11:00:07.261224+0000","last_active":"2024-04-25T11:00:08.259925+0000","last_peered":"2024-04-25T11:00:08.259925+0000","last_clean":"2024-04-25T11:00:08.259925+0000","last_became_active":"2024-04-25T11:00:07.260917+0000","last_became_peered":"2024-04-25T11:00:07.260917+0000","last_unstale":"2024-04-25T11:00:08.259925+0000","last_undegraded":"2024-04-25T11:00:08.259925+0000","last_fullsized":"2024-04-25T11:00:08.259925+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:50.570436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265046+0000","last_change":"2024-04-25T11:00:06.256448+0000","last_active":"2024-04-25T11:00:08.265046+0000","last_peered":"2024-04-25T11:00:08.265046+0000","last_clean":"2024-04-25T11:00:08.265046+0000","last_became_active":"2024-04-25T11:00:06.256095+0000","last_became_peered":"2024-04-25T11:00:06.256095+0000","last_unstale":"2024-04-25T11:00:08.265046+0000","last_undegraded":"2024-04-25T11:00:08.265046+0000","last_fullsized":"2024-04-25T11:00:08.265046+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:37.511626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835002+0000","last_change":"2024-04-25T11:00:07.263575+0000","last_active":"2024-04-25T11:00:08.835002+0000","last_peered":"2024-04-25T11:00:08.835002+0000","last_clean":"2024-04-25T11:00:08.835002+0000","last_became_active":"2024-04-25T11:00:07.263296+0000","last_became_peered":"2024-04-25T11:00:07.263296+0000","last_unstale":"2024-04-25T11:00:08.835002+0000","last_undegraded":"2024-04-25T11:00:08.835002+0000","last_fullsized":"2024-04-25T11:00:08.835002+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:24.519419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265630+0000","last_change":"2024-04-25T11:00:06.261291+0000","last_active":"2024-04-25T11:00:08.265630+0000","last_peered":"2024-04-25T11:00:08.265630+0000","last_clean":"2024-04-25T11:00:08.265630+0000","last_became_active":"2024-04-25T11:00:06.260969+0000","last_became_peered":"2024-04-25T11:00:06.260969+0000","last_unstale":"2024-04-25T11:00:08.265630+0000","last_undegraded":"2024-04-25T11:00:08.265630+0000","last_fullsized":"2024-04-25T11:00:08.265630+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:08.488342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.834714+0000","last_change":"2024-04-25T11:00:07.258332+0000","last_active":"2024-04-25T11:00:08.834714+0000","last_peered":"2024-04-25T11:00:08.834714+0000","last_clean":"2024-04-25T11:00:08.834714+0000","last_became_active":"2024-04-25T11:00:07.258010+0000","last_became_peered":"2024-04-25T11:00:07.258010+0000","last_unstale":"2024-04-25T11:00:08.834714+0000","last_undegraded":"2024-04-25T11:00:08.834714+0000","last_fullsized":"2024-04-25T11:00:08.834714+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:03.924681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.257049+0000","last_change":"2024-04-25T11:00:06.259265+0000","last_active":"2024-04-25T11:00:08.257049+0000","last_peered":"2024-04-25T11:00:08.257049+0000","last_clean":"2024-04-25T11:00:08.257049+0000","last_became_active":"2024-04-25T11:00:06.258949+0000","last_became_peered":"2024-04-25T11:00:06.258949+0000","last_unstale":"2024-04-25T11:00:08.257049+0000","last_undegraded":"2024-04-25T11:00:08.257049+0000","last_fullsized":"2024-04-25T11:00:08.257049+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:55.930698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.834983+0000","last_change":"2024-04-25T11:00:07.258792+0000","last_active":"2024-04-25T11:00:08.834983+0000","last_peered":"2024-04-25T11:00:08.834983+0000","last_clean":"2024-04-25T11:00:08.834983+0000","last_became_active":"2024-04-25T11:00:07.258502+0000","last_became_peered":"2024-04-25T11:00:07.258502+0000","last_unstale":"2024-04-25T11:00:08.834983+0000","last_undegraded":"2024-04-25T11:00:08.834983+0000","last_fullsized":"2024-04-25T11:00:08.834983+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:11.868608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.264585+0000","last_change":"2024-04-25T11:00:06.255869+0000","last_active":"2024-04-25T11:00:08.264585+0000","last_peered":"2024-04-25T11:00:08.264585+0000","last_clean":"2024-04-25T11:00:08.264585+0000","last_became_active":"2024-04-25T11:00:06.255468+0000","last_became_peered":"2024-04-25T11:00:06.255468+0000","last_unstale":"2024-04-25T11:00:08.264585+0000","last_undegraded":"2024-04-25T11:00:08.264585+0000","last_fullsized":"2024-04-25T11:00:08.264585+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:08.488342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.834579+0000","last_change":"2024-04-25T11:00:07.259018+0000","last_active":"2024-04-25T11:00:08.834579+0000","last_peered":"2024-04-25T11:00:08.834579+0000","last_clean":"2024-04-25T11:00:08.834579+0000","last_became_active":"2024-04-25T11:00:07.258712+0000","last_became_peered":"2024-04-25T11:00:07.258712+0000","last_unstale":"2024-04-25T11:00:08.834579+0000","last_undegraded":"2024-04-25T11:00:08.834579+0000","last_fullsized":"2024-04-25T11:00:08.834579+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:04.341143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265489+0000","last_change":"2024-04-25T11:00:06.259165+0000","last_active":"2024-04-25T11:00:08.265489+0000","last_peered":"2024-04-25T11:00:08.265489+0000","last_clean":"2024-04-25T11:00:08.265489+0000","last_became_active":"2024-04-25T11:00:06.258814+0000","last_became_peered":"2024-04-25T11:00:06.258814+0000","last_unstale":"2024-04-25T11:00:08.265489+0000","last_undegraded":"2024-04-25T11:00:08.265489+0000","last_fullsized":"2024-04-25T11:00:08.265489+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:40.478099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.834973+0000","last_change":"2024-04-25T11:00:07.262688+0000","last_active":"2024-04-25T11:00:08.834973+0000","last_peered":"2024-04-25T11:00:08.834973+0000","last_clean":"2024-04-25T11:00:08.834973+0000","last_became_active":"2024-04-25T11:00:07.262213+0000","last_became_peered":"2024-04-25T11:00:07.262213+0000","last_unstale":"2024-04-25T11:00:08.834973+0000","last_undegraded":"2024-04-25T11:00:08.834973+0000","last_fullsized":"2024-04-25T11:00:08.834973+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:51.247079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.333802+0000","last_change":"2024-04-25T11:00:06.266180+0000","last_active":"2024-04-25T11:00:08.333802+0000","last_peered":"2024-04-25T11:00:08.333802+0000","last_clean":"2024-04-25T11:00:08.333802+0000","last_became_active":"2024-04-25T11:00:06.265801+0000","last_became_peered":"2024-04-25T11:00:06.265801+0000","last_unstale":"2024-04-25T11:00:08.333802+0000","last_undegraded":"2024-04-25T11:00:08.333802+0000","last_fullsized":"2024-04-25T11:00:08.333802+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:16.726713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.834713+0000","last_change":"2024-04-25T11:00:07.260694+0000","last_active":"2024-04-25T11:00:08.834713+0000","last_peered":"2024-04-25T11:00:08.834713+0000","last_clean":"2024-04-25T11:00:08.834713+0000","last_became_active":"2024-04-25T11:00:07.260414+0000","last_became_peered":"2024-04-25T11:00:07.260414+0000","last_unstale":"2024-04-25T11:00:08.834713+0000","last_undegraded":"2024-04-25T11:00:08.834713+0000","last_fullsized":"2024-04-25T11:00:08.834713+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:43.915729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.260364+0000","last_change":"2024-04-25T11:00:06.257887+0000","last_active":"2024-04-25T11:00:08.260364+0000","last_peered":"2024-04-25T11:00:08.260364+0000","last_clean":"2024-04-25T11:00:08.260364+0000","last_became_active":"2024-04-25T11:00:06.257563+0000","last_became_peered":"2024-04-25T11:00:06.257563+0000","last_unstale":"2024-04-25T11:00:08.260364+0000","last_undegraded":"2024-04-25T11:00:08.260364+0000","last_fullsized":"2024-04-25T11:00:08.260364+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:55.930698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.334478+0000","last_change":"2024-04-25T11:00:07.271840+0000","last_active":"2024-04-25T11:00:08.334478+0000","last_peered":"2024-04-25T11:00:08.334478+0000","last_clean":"2024-04-25T11:00:08.334478+0000","last_became_active":"2024-04-25T11:00:07.269612+0000","last_became_peered":"2024-04-25T11:00:07.269612+0000","last_unstale":"2024-04-25T11:00:08.334478+0000","last_undegraded":"2024-04-25T11:00:08.334478+0000","last_fullsized":"2024-04-25T11:00:08.334478+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:07.367840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"31'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.257365+0000","last_change":"2024-04-25T11:00:06.259220+0000","last_active":"2024-04-25T11:00:08.257365+0000","last_peered":"2024-04-25T11:00:08.257365+0000","last_clean":"2024-04-25T11:00:08.257365+0000","last_became_active":"2024-04-25T11:00:06.258895+0000","last_became_peered":"2024-04-25T11:00:06.258895+0000","last_unstale":"2024-04-25T11:00:08.257365+0000","last_undegraded":"2024-04-25T11:00:08.257365+0000","last_fullsized":"2024-04-25T11:00:08.257365+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:10.861006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.256984+0000","last_change":"2024-04-25T11:00:07.271940+0000","last_active":"2024-04-25T11:00:08.256984+0000","last_peered":"2024-04-25T11:00:08.256984+0000","last_clean":"2024-04-25T11:00:08.256984+0000","last_became_active":"2024-04-25T11:00:07.271640+0000","last_became_peered":"2024-04-25T11:00:07.271640+0000","last_unstale":"2024-04-25T11:00:08.256984+0000","last_undegraded":"2024-04-25T11:00:08.256984+0000","last_fullsized":"2024-04-25T11:00:08.256984+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:00.515448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.259999+0000","last_change":"2024-04-25T11:00:06.257498+0000","last_active":"2024-04-25T11:00:08.259999+0000","last_peered":"2024-04-25T11:00:08.259999+0000","last_clean":"2024-04-25T11:00:08.259999+0000","last_became_active":"2024-04-25T11:00:06.257083+0000","last_became_peered":"2024-04-25T11:00:06.257083+0000","last_unstale":"2024-04-25T11:00:08.259999+0000","last_undegraded":"2024-04-25T11:00:08.259999+0000","last_fullsized":"2024-04-25T11:00:08.259999+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:37.048525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.264607+0000","last_change":"2024-04-25T11:00:07.254833+0000","last_active":"2024-04-25T11:00:08.264607+0000","last_peered":"2024-04-25T11:00:08.264607+0000","last_clean":"2024-04-25T11:00:08.264607+0000","last_became_active":"2024-04-25T11:00:07.254472+0000","last_became_peered":"2024-04-25T11:00:07.254472+0000","last_unstale":"2024-04-25T11:00:08.264607+0000","last_undegraded":"2024-04-25T11:00:08.264607+0000","last_fullsized":"2024-04-25T11:00:08.264607+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:11.868608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.257207+0000","last_change":"2024-04-25T11:00:06.257789+0000","last_active":"2024-04-25T11:00:08.257207+0000","last_peered":"2024-04-25T11:00:08.257207+0000","last_clean":"2024-04-25T11:00:08.257207+0000","last_became_active":"2024-04-25T11:00:06.257450+0000","last_became_peered":"2024-04-25T11:00:06.257450+0000","last_unstale":"2024-04-25T11:00:08.257207+0000","last_undegraded":"2024-04-25T11:00:08.257207+0000","last_fullsized":"2024-04-25T11:00:08.257207+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:42.908127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.257067+0000","last_change":"2024-04-25T11:00:07.255875+0000","last_active":"2024-04-25T11:00:08.257067+0000","last_peered":"2024-04-25T11:00:08.257067+0000","last_clean":"2024-04-25T11:00:08.257067+0000","last_became_active":"2024-04-25T11:00:07.255565+0000","last_became_peered":"2024-04-25T11:00:07.255565+0000","last_unstale":"2024-04-25T11:00:08.257067+0000","last_undegraded":"2024-04-25T11:00:08.257067+0000","last_fullsized":"2024-04-25T11:00:08.257067+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:51.748199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.259845+0000","last_change":"2024-04-25T11:00:06.257884+0000","last_active":"2024-04-25T11:00:08.259845+0000","last_peered":"2024-04-25T11:00:08.259845+0000","last_clean":"2024-04-25T11:00:08.259845+0000","last_became_active":"2024-04-25T11:00:06.257586+0000","last_became_peered":"2024-04-25T11:00:06.257586+0000","last_unstale":"2024-04-25T11:00:08.259845+0000","last_undegraded":"2024-04-25T11:00:08.259845+0000","last_fullsized":"2024-04-25T11:00:08.259845+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:42.908127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.834496+0000","last_change":"2024-04-25T11:00:07.274001+0000","last_active":"2024-04-25T11:00:08.834496+0000","last_peered":"2024-04-25T11:00:08.834496+0000","last_clean":"2024-04-25T11:00:08.834496+0000","last_became_active":"2024-04-25T11:00:07.273698+0000","last_became_peered":"2024-04-25T11:00:07.273698+0000","last_unstale":"2024-04-25T11:00:08.834496+0000","last_undegraded":"2024-04-25T11:00:08.834496+0000","last_fullsized":"2024-04-25T11:00:08.834496+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:18.962985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265502+0000","last_change":"2024-04-25T11:00:06.258316+0000","last_active":"2024-04-25T11:00:08.265502+0000","last_peered":"2024-04-25T11:00:08.265502+0000","last_clean":"2024-04-25T11:00:08.265502+0000","last_became_active":"2024-04-25T11:00:06.257838+0000","last_became_peered":"2024-04-25T11:00:06.257838+0000","last_unstale":"2024-04-25T11:00:08.265502+0000","last_undegraded":"2024-04-25T11:00:08.265502+0000","last_fullsized":"2024-04-25T11:00:08.265502+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:44.176669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265356+0000","last_change":"2024-04-25T11:00:07.268312+0000","last_active":"2024-04-25T11:00:08.265356+0000","last_peered":"2024-04-25T11:00:08.265356+0000","last_clean":"2024-04-25T11:00:08.265356+0000","last_became_active":"2024-04-25T11:00:07.267952+0000","last_became_peered":"2024-04-25T11:00:07.267952+0000","last_unstale":"2024-04-25T11:00:08.265356+0000","last_undegraded":"2024-04-25T11:00:08.265356+0000","last_fullsized":"2024-04-25T11:00:08.265356+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:51.748199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.834608+0000","last_change":"2024-04-25T11:00:06.266105+0000","last_active":"2024-04-25T11:00:08.834608+0000","last_peered":"2024-04-25T11:00:08.834608+0000","last_clean":"2024-04-25T11:00:08.834608+0000","last_became_active":"2024-04-25T11:00:06.265786+0000","last_became_peered":"2024-04-25T11:00:06.265786+0000","last_unstale":"2024-04-25T11:00:08.834608+0000","last_undegraded":"2024-04-25T11:00:08.834608+0000","last_fullsized":"2024-04-25T11:00:08.834608+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:27.813350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835213+0000","last_change":"2024-04-25T11:00:07.265240+0000","last_active":"2024-04-25T11:00:08.835213+0000","last_peered":"2024-04-25T11:00:08.835213+0000","last_clean":"2024-04-25T11:00:08.835213+0000","last_became_active":"2024-04-25T11:00:07.264915+0000","last_became_peered":"2024-04-25T11:00:07.264915+0000","last_unstale":"2024-04-25T11:00:08.835213+0000","last_undegraded":"2024-04-25T11:00:08.835213+0000","last_fullsized":"2024-04-25T11:00:08.835213+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:16.167953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.264616+0000","last_change":"2024-04-25T11:00:06.250003+0000","last_active":"2024-04-25T11:00:08.264616+0000","last_peered":"2024-04-25T11:00:08.264616+0000","last_clean":"2024-04-25T11:00:08.264616+0000","last_became_active":"2024-04-25T11:00:06.249632+0000","last_became_peered":"2024-04-25T11:00:06.249632+0000","last_unstale":"2024-04-25T11:00:08.264616+0000","last_undegraded":"2024-04-25T11:00:08.264616+0000","last_fullsized":"2024-04-25T11:00:08.264616+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:11.002336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":129,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835561+0000","last_change":"2024-04-25T10:59:07.839466+0000","last_active":"2024-04-25T11:00:08.835561+0000","last_peered":"2024-04-25T11:00:08.835561+0000","last_clean":"2024-04-25T11:00:08.835561+0000","last_became_active":"2024-04-25T10:59:07.836269+0000","last_became_peered":"2024-04-25T10:59:07.836269+0000","last_unstale":"2024-04-25T11:00:08.835561+0000","last_undegraded":"2024-04-25T11:00:08.835561+0000","last_fullsized":"2024-04-25T11:00:08.835561+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:06.549663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:06.549663+0000","last_clean_scrub_stamp":"2024-04-25T10:59:06.549663+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:21.575330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":946176,"data_stored":724908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":356352,"data_stored":135066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":356352,"data_stored":135122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":356352,"data_stored":134554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":364544,"data_stored":136084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":352256,"data_stored":134520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":364544,"data_stored":136084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":942080,"data_stored":724818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:00:15.702 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:00:15.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.821+0000 7fe14c4bd640 1 Processor -- start 2024-04-25T11:00:15.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.821+0000 7fe14c4bd640 1 -- start start 2024-04-25T11:00:15.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.823+0000 7fe14c4bd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe1440d1850 0x7fe1440cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:15.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.823+0000 7fe14c4bd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe14406d7e0 0x7fe14406dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:15.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.823+0000 7fe14c4bd640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe14406e130 con 0x7fe1440b7750 2024-04-25T11:00:15.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.823+0000 7fe14c4bd640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fe1440b3af0 con 0x7fe1440d1850 2024-04-25T11:00:15.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.823+0000 7fe14c4bd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe144164320 con 0x7fe14406d7e0 2024-04-25T11:00:15.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.823+0000 7fe14aa5a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe14406d7e0 0x7fe14406dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:15.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.823+0000 7fe149a58640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe1440d1850 0x7fe1440cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:15.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.823+0000 7fe14a259640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fe1440b7750 0x7fe144164800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.69:53962/0 (socket says 172.21.15.69:53962) 2024-04-25T11:00:15.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.823+0000 7fe14a259640 1 -- 172.21.15.69:0/2599081860 learned_addr learned my addr 172.21.15.69:0/2599081860 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:15.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.823+0000 7fe14aa5a640 1 -- 172.21.15.69:0/2599081860 >> v2:172.21.15.146:3300/0 conn(0x7fe1440d1850 msgr2=0x7fe1440cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:15.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.823+0000 7fe14aa5a640 1 --2- 172.21.15.69:0/2599081860 >> v2:172.21.15.146:3300/0 conn(0x7fe1440d1850 0x7fe1440cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:15.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.823+0000 7fe14aa5a640 1 -- 172.21.15.69:0/2599081860 >> v1:172.21.15.146:6789/0 conn(0x7fe1440b7750 legacy=0x7fe144164800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:15.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.823+0000 7fe14aa5a640 1 -- 172.21.15.69:0/2599081860 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1440b2f70 con 0x7fe14406d7e0 2024-04-25T11:00:15.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.823+0000 7fe149a58640 1 --2- 172.21.15.69:0/2599081860 >> v2:172.21.15.146:3300/0 conn(0x7fe1440d1850 0x7fe1440cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:00:15.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.824+0000 7fe14aa5a640 1 --2- 172.21.15.69:0/2599081860 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe14406d7e0 0x7fe14406dbf0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe13c0097a0 tx=0x7fe13c02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6995241dd6993da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:15.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.824+0000 7fe149257640 1 -- 172.21.15.69:0/2599081860 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c037070 con 0x7fe14406d7e0 2024-04-25T11:00:15.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.824+0000 7fe149257640 1 -- 172.21.15.69:0/2599081860 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe13c02ae50 con 0x7fe14406d7e0 2024-04-25T11:00:15.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.824+0000 7fe149257640 1 -- 172.21.15.69:0/2599081860 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c032660 con 0x7fe14406d7e0 2024-04-25T11:00:15.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.825+0000 7fe14c4bd640 1 -- 172.21.15.69:0/2599081860 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe14406d7e0 msgr2=0x7fe14406dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:15.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.825+0000 7fe14c4bd640 1 --2- 172.21.15.69:0/2599081860 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe14406d7e0 0x7fe14406dbf0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe13c0097a0 tx=0x7fe13c02a100 comp rx=0 tx=0).stop 2024-04-25T11:00:15.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.825+0000 7fe14c4bd640 1 -- 172.21.15.69:0/2599081860 shutdown_connections 2024-04-25T11:00:15.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.825+0000 7fe14c4bd640 1 --2- 172.21.15.69:0/2599081860 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe14406d7e0 0x7fe14406dbf0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:15.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.825+0000 7fe14c4bd640 1 --2- 172.21.15.69:0/2599081860 >> v2:172.21.15.146:3300/0 conn(0x7fe1440d1850 0x7fe1440cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:15.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.825+0000 7fe14c4bd640 1 -- 172.21.15.69:0/2599081860 >> 172.21.15.69:0/2599081860 conn(0x7fe14409b7d0 msgr2=0x7fe1440b7040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:15.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.825+0000 7fe14c4bd640 1 -- 172.21.15.69:0/2599081860 shutdown_connections 2024-04-25T11:00:15.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.825+0000 7fe14c4bd640 1 -- 172.21.15.69:0/2599081860 wait complete. 2024-04-25T11:00:15.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.827+0000 7fe14c4bd640 1 Processor -- start 2024-04-25T11:00:15.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.827+0000 7fe14c4bd640 1 -- start start 2024-04-25T11:00:15.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.828+0000 7fe14c4bd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe14406d7e0 0x7fe144163870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:15.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.828+0000 7fe14c4bd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1440b7750 0x7fe14409e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:15.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.828+0000 7fe14c4bd640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1440d1850 0x7fe144070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:15.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.828+0000 7fe14c4bd640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe1440b3810 con 0x7fe14406d7e0 2024-04-25T11:00:15.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.828+0000 7fe14c4bd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe1440b3530 con 0x7fe1440b7750 2024-04-25T11:00:15.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.828+0000 7fe14c4bd640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fe144164f10 con 0x7fe1440d1850 2024-04-25T11:00:15.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.828+0000 7fe14a259640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe14406d7e0 0x7fe144163870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:15.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.828+0000 7fe14a259640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe14406d7e0 0x7fe144163870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51012/0 (socket says 172.21.15.69:51012) 2024-04-25T11:00:15.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.828+0000 7fe14a259640 1 -- 172.21.15.69:0/411723445 learned_addr learned my addr 172.21.15.69:0/411723445 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:15.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.829+0000 7fe149a58640 1 --2- 172.21.15.69:0/411723445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1440b7750 0x7fe14409e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:15.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.829+0000 7fe14aa5a640 1 --2- 172.21.15.69:0/411723445 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1440d1850 0x7fe144070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:15.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.829+0000 7fe14a259640 1 -- 172.21.15.69:0/411723445 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1440d1850 msgr2=0x7fe144070ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:15.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.829+0000 7fe14a259640 1 --2- 172.21.15.69:0/411723445 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1440d1850 0x7fe144070ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:15.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.829+0000 7fe14a259640 1 -- 172.21.15.69:0/411723445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1440b7750 msgr2=0x7fe14409e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:15.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.829+0000 7fe14a259640 1 --2- 172.21.15.69:0/411723445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1440b7750 0x7fe14409e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:15.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.829+0000 7fe14a259640 1 -- 172.21.15.69:0/411723445 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe14409edc0 con 0x7fe14406d7e0 2024-04-25T11:00:15.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.829+0000 7fe14a259640 1 --2- 172.21.15.69:0/411723445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe14406d7e0 0x7fe144163870 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe14000cad0 tx=0x7fe14000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:15.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.829+0000 7fe13affd640 1 -- 172.21.15.69:0/411723445 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe140017070 con 0x7fe14406d7e0 2024-04-25T11:00:15.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.829+0000 7fe13affd640 1 -- 172.21.15.69:0/411723445 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe14000abd0 con 0x7fe14406d7e0 2024-04-25T11:00:15.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.829+0000 7fe14c4bd640 1 -- 172.21.15.69:0/411723445 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe14409f0a0 con 0x7fe14406d7e0 2024-04-25T11:00:15.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.829+0000 7fe13affd640 1 -- 172.21.15.69:0/411723445 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe140012400 con 0x7fe14406d7e0 2024-04-25T11:00:15.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.829+0000 7fe14c4bd640 1 -- 172.21.15.69:0/411723445 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe14409f660 con 0x7fe14406d7e0 2024-04-25T11:00:15.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.830+0000 7fe14c4bd640 1 -- 172.21.15.69:0/411723445 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe144050740 con 0x7fe14406d7e0 2024-04-25T11:00:15.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.831+0000 7fe13affd640 1 -- 172.21.15.69:0/411723445 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe14001a070 con 0x7fe14406d7e0 2024-04-25T11:00:15.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.831+0000 7fe13affd640 1 --2- 172.21.15.69:0/411723445 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe1280744e0 0x7fe1280769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:15.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.831+0000 7fe149a58640 1 --2- 172.21.15.69:0/411723445 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe1280744e0 0x7fe1280769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:15.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.832+0000 7fe13affd640 1 -- 172.21.15.69:0/411723445 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fe140094ee0 con 0x7fe14406d7e0 2024-04-25T11:00:15.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.832+0000 7fe149a58640 1 --2- 172.21.15.69:0/411723445 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe1280744e0 0x7fe1280769f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe1441613e0 tx=0x7fe134008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:15.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:15.836+0000 7fe13affd640 1 -- 172.21.15.69:0/411723445 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe14406e130 con 0x7fe14406d7e0 2024-04-25T11:00:16.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.005+0000 7fe14c4bd640 1 -- 172.21.15.69:0/411723445 --> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe1440d2f00 con 0x7fe1280744e0 2024-04-25T11:00:16.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.011+0000 7fe13affd640 1 -- 172.21.15.69:0/411723445 <== mgr.4107 v2:172.21.15.69:6832/1107421267 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251944 (secure 0 0 0) 0x7fe1440d2f00 con 0x7fe1280744e0 2024-04-25T11:00:16.014 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:00:16.015 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-04-25T11:00:16.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.014+0000 7fe14c4bd640 1 -- 172.21.15.69:0/411723445 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe1280744e0 msgr2=0x7fe1280769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:16.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.014+0000 7fe14c4bd640 1 --2- 172.21.15.69:0/411723445 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe1280744e0 0x7fe1280769f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe1441613e0 tx=0x7fe134008040 comp rx=0 tx=0).stop 2024-04-25T11:00:16.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.014+0000 7fe14c4bd640 1 -- 172.21.15.69:0/411723445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe14406d7e0 msgr2=0x7fe144163870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:16.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.014+0000 7fe14c4bd640 1 --2- 172.21.15.69:0/411723445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe14406d7e0 0x7fe144163870 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe14000cad0 tx=0x7fe14000cf90 comp rx=0 tx=0).stop 2024-04-25T11:00:16.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.014+0000 7fe14c4bd640 1 -- 172.21.15.69:0/411723445 shutdown_connections 2024-04-25T11:00:16.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.014+0000 7fe14c4bd640 1 --2- 172.21.15.69:0/411723445 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fe1280744e0 0x7fe1280769f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:16.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.014+0000 7fe14c4bd640 1 --2- 172.21.15.69:0/411723445 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1440d1850 0x7fe144070ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:16.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.014+0000 7fe14c4bd640 1 --2- 172.21.15.69:0/411723445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1440b7750 0x7fe14409e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:16.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.014+0000 7fe14c4bd640 1 --2- 172.21.15.69:0/411723445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe14406d7e0 0x7fe144163870 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:16.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.014+0000 7fe14c4bd640 1 -- 172.21.15.69:0/411723445 >> 172.21.15.69:0/411723445 conn(0x7fe14409b7d0 msgr2=0x7fe1440d22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:16.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.014+0000 7fe14c4bd640 1 -- 172.21.15.69:0/411723445 shutdown_connections 2024-04-25T11:00:16.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.014+0000 7fe14c4bd640 1 -- 172.21.15.69:0/411723445 wait complete. 2024-04-25T11:00:16.026 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2024-04-25T11:00:14.416877+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":177,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220148,"kb_used_data":3944,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754572,"statfs":{"total":773094113280,"available":772868681728,"internally_reserved":0,"allocated":4038656,"data_stored":2261156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001702"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265179+0000","last_change":"2024-04-25T11:00:07.262796+0000","last_active":"2024-04-25T11:00:08.265179+0000","last_peered":"2024-04-25T11:00:08.265179+0000","last_clean":"2024-04-25T11:00:08.265179+0000","last_became_active":"2024-04-25T11:00:07.262396+0000","last_became_peered":"2024-04-25T11:00:07.262396+0000","last_unstale":"2024-04-25T11:00:08.265179+0000","last_undegraded":"2024-04-25T11:00:08.265179+0000","last_fullsized":"2024-04-25T11:00:08.265179+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:46.074685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835725+0000","last_change":"2024-04-25T11:00:07.258056+0000","last_active":"2024-04-25T11:00:08.835725+0000","last_peered":"2024-04-25T11:00:08.835725+0000","last_clean":"2024-04-25T11:00:08.835725+0000","last_became_active":"2024-04-25T11:00:07.257741+0000","last_became_peered":"2024-04-25T11:00:07.257741+0000","last_unstale":"2024-04-25T11:00:08.835725+0000","last_undegraded":"2024-04-25T11:00:08.835725+0000","last_fullsized":"2024-04-25T11:00:08.835725+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:30.196237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.334119+0000","last_change":"2024-04-25T11:00:07.270241+0000","last_active":"2024-04-25T11:00:08.334119+0000","last_peered":"2024-04-25T11:00:08.334119+0000","last_clean":"2024-04-25T11:00:08.334119+0000","last_became_active":"2024-04-25T11:00:07.269930+0000","last_became_peered":"2024-04-25T11:00:07.269930+0000","last_unstale":"2024-04-25T11:00:08.334119+0000","last_undegraded":"2024-04-25T11:00:08.334119+0000","last_fullsized":"2024-04-25T11:00:08.334119+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:03.486987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.259807+0000","last_change":"2024-04-25T11:00:07.261375+0000","last_active":"2024-04-25T11:00:08.259807+0000","last_peered":"2024-04-25T11:00:08.259807+0000","last_clean":"2024-04-25T11:00:08.259807+0000","last_became_active":"2024-04-25T11:00:07.261092+0000","last_became_peered":"2024-04-25T11:00:07.261092+0000","last_unstale":"2024-04-25T11:00:08.259807+0000","last_undegraded":"2024-04-25T11:00:08.259807+0000","last_fullsized":"2024-04-25T11:00:08.259807+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:20.203175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.834594+0000","last_change":"2024-04-25T11:00:07.266720+0000","last_active":"2024-04-25T11:00:08.834594+0000","last_peered":"2024-04-25T11:00:08.834594+0000","last_clean":"2024-04-25T11:00:08.834594+0000","last_became_active":"2024-04-25T11:00:07.266441+0000","last_became_peered":"2024-04-25T11:00:07.266441+0000","last_unstale":"2024-04-25T11:00:08.834594+0000","last_undegraded":"2024-04-25T11:00:08.834594+0000","last_fullsized":"2024-04-25T11:00:08.834594+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:56.938301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835056+0000","last_change":"2024-04-25T11:00:07.266161+0000","last_active":"2024-04-25T11:00:08.835056+0000","last_peered":"2024-04-25T11:00:08.835056+0000","last_clean":"2024-04-25T11:00:08.835056+0000","last_became_active":"2024-04-25T11:00:07.265871+0000","last_became_peered":"2024-04-25T11:00:07.265871+0000","last_unstale":"2024-04-25T11:00:08.835056+0000","last_undegraded":"2024-04-25T11:00:08.835056+0000","last_fullsized":"2024-04-25T11:00:08.835056+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:40.917365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835236+0000","last_change":"2024-04-25T11:00:07.266379+0000","last_active":"2024-04-25T11:00:08.835236+0000","last_peered":"2024-04-25T11:00:08.835236+0000","last_clean":"2024-04-25T11:00:08.835236+0000","last_became_active":"2024-04-25T11:00:07.266022+0000","last_became_peered":"2024-04-25T11:00:07.266022+0000","last_unstale":"2024-04-25T11:00:08.835236+0000","last_undegraded":"2024-04-25T11:00:08.835236+0000","last_fullsized":"2024-04-25T11:00:08.835236+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:30.196237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.834900+0000","last_change":"2024-04-25T11:00:07.264408+0000","last_active":"2024-04-25T11:00:08.834900+0000","last_peered":"2024-04-25T11:00:08.834900+0000","last_clean":"2024-04-25T11:00:08.834900+0000","last_became_active":"2024-04-25T11:00:07.264015+0000","last_became_peered":"2024-04-25T11:00:07.264015+0000","last_unstale":"2024-04-25T11:00:08.834900+0000","last_undegraded":"2024-04-25T11:00:08.834900+0000","last_fullsized":"2024-04-25T11:00:08.834900+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:40.917365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265879+0000","last_change":"2024-04-25T11:00:07.270707+0000","last_active":"2024-04-25T11:00:08.265879+0000","last_peered":"2024-04-25T11:00:08.265879+0000","last_clean":"2024-04-25T11:00:08.265879+0000","last_became_active":"2024-04-25T11:00:07.267820+0000","last_became_peered":"2024-04-25T11:00:07.267820+0000","last_unstale":"2024-04-25T11:00:08.265879+0000","last_undegraded":"2024-04-25T11:00:08.265879+0000","last_fullsized":"2024-04-25T11:00:08.265879+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:21.078922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.260109+0000","last_change":"2024-04-25T11:00:07.271255+0000","last_active":"2024-04-25T11:00:08.260109+0000","last_peered":"2024-04-25T11:00:08.260109+0000","last_clean":"2024-04-25T11:00:08.260109+0000","last_became_active":"2024-04-25T11:00:07.270847+0000","last_became_peered":"2024-04-25T11:00:07.270847+0000","last_unstale":"2024-04-25T11:00:08.260109+0000","last_undegraded":"2024-04-25T11:00:08.260109+0000","last_fullsized":"2024-04-25T11:00:08.260109+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:30.194502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835431+0000","last_change":"2024-04-25T11:00:07.270525+0000","last_active":"2024-04-25T11:00:08.835431+0000","last_peered":"2024-04-25T11:00:08.835431+0000","last_clean":"2024-04-25T11:00:08.835431+0000","last_became_active":"2024-04-25T11:00:07.270208+0000","last_became_peered":"2024-04-25T11:00:07.270208+0000","last_unstale":"2024-04-25T11:00:08.835431+0000","last_undegraded":"2024-04-25T11:00:08.835431+0000","last_fullsized":"2024-04-25T11:00:08.835431+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:07.367840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.334090+0000","last_change":"2024-04-25T11:00:07.265019+0000","last_active":"2024-04-25T11:00:08.334090+0000","last_peered":"2024-04-25T11:00:08.334090+0000","last_clean":"2024-04-25T11:00:08.334090+0000","last_became_active":"2024-04-25T11:00:07.264666+0000","last_became_peered":"2024-04-25T11:00:07.264666+0000","last_unstale":"2024-04-25T11:00:08.334090+0000","last_undegraded":"2024-04-25T11:00:08.334090+0000","last_fullsized":"2024-04-25T11:00:08.334090+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:51.247079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.834823+0000","last_change":"2024-04-25T11:00:07.267683+0000","last_active":"2024-04-25T11:00:08.834823+0000","last_peered":"2024-04-25T11:00:08.834823+0000","last_clean":"2024-04-25T11:00:08.834823+0000","last_became_active":"2024-04-25T11:00:07.267400+0000","last_became_peered":"2024-04-25T11:00:07.267400+0000","last_unstale":"2024-04-25T11:00:08.834823+0000","last_undegraded":"2024-04-25T11:00:08.834823+0000","last_fullsized":"2024-04-25T11:00:08.834823+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:48.605487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835217+0000","last_change":"2024-04-25T11:00:07.265397+0000","last_active":"2024-04-25T11:00:08.835217+0000","last_peered":"2024-04-25T11:00:08.835217+0000","last_clean":"2024-04-25T11:00:08.835217+0000","last_became_active":"2024-04-25T11:00:07.264141+0000","last_became_peered":"2024-04-25T11:00:07.264141+0000","last_unstale":"2024-04-25T11:00:08.835217+0000","last_undegraded":"2024-04-25T11:00:08.835217+0000","last_fullsized":"2024-04-25T11:00:08.835217+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:51.247079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.334023+0000","last_change":"2024-04-25T11:00:07.266066+0000","last_active":"2024-04-25T11:00:08.334023+0000","last_peered":"2024-04-25T11:00:08.334023+0000","last_clean":"2024-04-25T11:00:08.334023+0000","last_became_active":"2024-04-25T11:00:07.264905+0000","last_became_peered":"2024-04-25T11:00:07.264905+0000","last_unstale":"2024-04-25T11:00:08.334023+0000","last_undegraded":"2024-04-25T11:00:08.334023+0000","last_fullsized":"2024-04-25T11:00:08.334023+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:40.917365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835627+0000","last_change":"2024-04-25T11:00:07.273989+0000","last_active":"2024-04-25T11:00:08.835627+0000","last_peered":"2024-04-25T11:00:08.835627+0000","last_clean":"2024-04-25T11:00:08.835627+0000","last_became_active":"2024-04-25T11:00:07.273658+0000","last_became_peered":"2024-04-25T11:00:07.273658+0000","last_unstale":"2024-04-25T11:00:08.835627+0000","last_undegraded":"2024-04-25T11:00:08.835627+0000","last_fullsized":"2024-04-25T11:00:08.835627+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:10.131991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835306+0000","last_change":"2024-04-25T11:00:07.267254+0000","last_active":"2024-04-25T11:00:08.835306+0000","last_peered":"2024-04-25T11:00:08.835306+0000","last_clean":"2024-04-25T11:00:08.835306+0000","last_became_active":"2024-04-25T11:00:07.266905+0000","last_became_peered":"2024-04-25T11:00:07.266905+0000","last_unstale":"2024-04-25T11:00:08.835306+0000","last_undegraded":"2024-04-25T11:00:08.835306+0000","last_fullsized":"2024-04-25T11:00:08.835306+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:09.261812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835790+0000","last_change":"2024-04-25T11:00:07.267161+0000","last_active":"2024-04-25T11:00:08.835790+0000","last_peered":"2024-04-25T11:00:08.835790+0000","last_clean":"2024-04-25T11:00:08.835790+0000","last_became_active":"2024-04-25T11:00:07.266877+0000","last_became_peered":"2024-04-25T11:00:07.266877+0000","last_unstale":"2024-04-25T11:00:08.835790+0000","last_undegraded":"2024-04-25T11:00:08.835790+0000","last_fullsized":"2024-04-25T11:00:08.835790+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:09.261812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.834850+0000","last_change":"2024-04-25T11:00:07.273250+0000","last_active":"2024-04-25T11:00:08.834850+0000","last_peered":"2024-04-25T11:00:08.834850+0000","last_clean":"2024-04-25T11:00:08.834850+0000","last_became_active":"2024-04-25T11:00:07.272949+0000","last_became_peered":"2024-04-25T11:00:07.272949+0000","last_unstale":"2024-04-25T11:00:08.834850+0000","last_undegraded":"2024-04-25T11:00:08.834850+0000","last_fullsized":"2024-04-25T11:00:08.834850+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:38.615241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835058+0000","last_change":"2024-04-25T11:00:07.261730+0000","last_active":"2024-04-25T11:00:08.835058+0000","last_peered":"2024-04-25T11:00:08.835058+0000","last_clean":"2024-04-25T11:00:08.835058+0000","last_became_active":"2024-04-25T11:00:07.261415+0000","last_became_peered":"2024-04-25T11:00:07.261415+0000","last_unstale":"2024-04-25T11:00:08.835058+0000","last_undegraded":"2024-04-25T11:00:08.835058+0000","last_fullsized":"2024-04-25T11:00:08.835058+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:28.820953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265693+0000","last_change":"2024-04-25T11:00:07.258119+0000","last_active":"2024-04-25T11:00:08.265693+0000","last_peered":"2024-04-25T11:00:08.265693+0000","last_clean":"2024-04-25T11:00:08.265693+0000","last_became_active":"2024-04-25T11:00:07.257207+0000","last_became_peered":"2024-04-25T11:00:07.257207+0000","last_unstale":"2024-04-25T11:00:08.265693+0000","last_undegraded":"2024-04-25T11:00:08.265693+0000","last_fullsized":"2024-04-25T11:00:08.265693+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:50.570436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265357+0000","last_change":"2024-04-25T11:00:07.259122+0000","last_active":"2024-04-25T11:00:08.265357+0000","last_peered":"2024-04-25T11:00:08.265357+0000","last_clean":"2024-04-25T11:00:08.265357+0000","last_became_active":"2024-04-25T11:00:07.258787+0000","last_became_peered":"2024-04-25T11:00:07.258787+0000","last_unstale":"2024-04-25T11:00:08.265357+0000","last_undegraded":"2024-04-25T11:00:08.265357+0000","last_fullsized":"2024-04-25T11:00:08.265357+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:56.469604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.256876+0000","last_change":"2024-04-25T11:00:07.259818+0000","last_active":"2024-04-25T11:00:08.256876+0000","last_peered":"2024-04-25T11:00:08.256876+0000","last_clean":"2024-04-25T11:00:08.256876+0000","last_became_active":"2024-04-25T11:00:07.259187+0000","last_became_peered":"2024-04-25T11:00:07.259187+0000","last_unstale":"2024-04-25T11:00:08.256876+0000","last_undegraded":"2024-04-25T11:00:08.256876+0000","last_fullsized":"2024-04-25T11:00:08.256876+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:17.734316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.334087+0000","last_change":"2024-04-25T11:00:07.267648+0000","last_active":"2024-04-25T11:00:08.334087+0000","last_peered":"2024-04-25T11:00:08.334087+0000","last_clean":"2024-04-25T11:00:08.334087+0000","last_became_active":"2024-04-25T11:00:07.267349+0000","last_became_peered":"2024-04-25T11:00:07.267349+0000","last_unstale":"2024-04-25T11:00:08.334087+0000","last_undegraded":"2024-04-25T11:00:08.334087+0000","last_fullsized":"2024-04-25T11:00:08.334087+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:30.196237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.260262+0000","last_change":"2024-04-25T11:00:07.266306+0000","last_active":"2024-04-25T11:00:08.260262+0000","last_peered":"2024-04-25T11:00:08.260262+0000","last_clean":"2024-04-25T11:00:08.260262+0000","last_became_active":"2024-04-25T11:00:07.265935+0000","last_became_peered":"2024-04-25T11:00:07.265935+0000","last_unstale":"2024-04-25T11:00:08.260262+0000","last_undegraded":"2024-04-25T11:00:08.260262+0000","last_fullsized":"2024-04-25T11:00:08.260262+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:04.418320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835817+0000","last_change":"2024-04-25T11:00:07.262516+0000","last_active":"2024-04-25T11:00:08.835817+0000","last_peered":"2024-04-25T11:00:08.835817+0000","last_clean":"2024-04-25T11:00:08.835817+0000","last_became_active":"2024-04-25T11:00:07.261840+0000","last_became_peered":"2024-04-25T11:00:07.261840+0000","last_unstale":"2024-04-25T11:00:08.835817+0000","last_undegraded":"2024-04-25T11:00:08.835817+0000","last_fullsized":"2024-04-25T11:00:08.835817+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:19.428784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.260256+0000","last_change":"2024-04-25T11:00:07.265227+0000","last_active":"2024-04-25T11:00:08.260256+0000","last_peered":"2024-04-25T11:00:08.260256+0000","last_clean":"2024-04-25T11:00:08.260256+0000","last_became_active":"2024-04-25T11:00:07.264946+0000","last_became_peered":"2024-04-25T11:00:07.264946+0000","last_unstale":"2024-04-25T11:00:08.260256+0000","last_undegraded":"2024-04-25T11:00:08.260256+0000","last_fullsized":"2024-04-25T11:00:08.260256+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:21.078922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.334339+0000","last_change":"2024-04-25T11:00:07.259601+0000","last_active":"2024-04-25T11:00:08.334339+0000","last_peered":"2024-04-25T11:00:08.334339+0000","last_clean":"2024-04-25T11:00:08.334339+0000","last_became_active":"2024-04-25T11:00:07.258529+0000","last_became_peered":"2024-04-25T11:00:07.258529+0000","last_unstale":"2024-04-25T11:00:08.334339+0000","last_undegraded":"2024-04-25T11:00:08.334339+0000","last_fullsized":"2024-04-25T11:00:08.334339+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:09.261812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265850+0000","last_change":"2024-04-25T11:00:07.266448+0000","last_active":"2024-04-25T11:00:08.265850+0000","last_peered":"2024-04-25T11:00:08.265850+0000","last_clean":"2024-04-25T11:00:08.265850+0000","last_became_active":"2024-04-25T11:00:07.266118+0000","last_became_peered":"2024-04-25T11:00:07.266118+0000","last_unstale":"2024-04-25T11:00:08.265850+0000","last_undegraded":"2024-04-25T11:00:08.265850+0000","last_fullsized":"2024-04-25T11:00:08.265850+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:30.194502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835040+0000","last_change":"2024-04-25T11:00:07.266442+0000","last_active":"2024-04-25T11:00:08.835040+0000","last_peered":"2024-04-25T11:00:08.835040+0000","last_clean":"2024-04-25T11:00:08.835040+0000","last_became_active":"2024-04-25T11:00:07.266101+0000","last_became_peered":"2024-04-25T11:00:07.266101+0000","last_unstale":"2024-04-25T11:00:08.835040+0000","last_undegraded":"2024-04-25T11:00:08.835040+0000","last_fullsized":"2024-04-25T11:00:08.835040+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:03.486987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.264808+0000","last_change":"2024-04-25T11:00:07.260993+0000","last_active":"2024-04-25T11:00:08.264808+0000","last_peered":"2024-04-25T11:00:08.264808+0000","last_clean":"2024-04-25T11:00:08.264808+0000","last_became_active":"2024-04-25T11:00:07.260408+0000","last_became_peered":"2024-04-25T11:00:07.260408+0000","last_unstale":"2024-04-25T11:00:08.264808+0000","last_undegraded":"2024-04-25T11:00:08.264808+0000","last_fullsized":"2024-04-25T11:00:08.264808+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:24.519419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.834885+0000","last_change":"2024-04-25T11:00:07.272840+0000","last_active":"2024-04-25T11:00:08.834885+0000","last_peered":"2024-04-25T11:00:08.834885+0000","last_clean":"2024-04-25T11:00:08.834885+0000","last_became_active":"2024-04-25T11:00:07.272454+0000","last_became_peered":"2024-04-25T11:00:07.272454+0000","last_unstale":"2024-04-25T11:00:08.834885+0000","last_undegraded":"2024-04-25T11:00:08.834885+0000","last_fullsized":"2024-04-25T11:00:08.834885+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:38.615241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.257217+0000","last_change":"2024-04-25T11:00:07.258519+0000","last_active":"2024-04-25T11:00:08.257217+0000","last_peered":"2024-04-25T11:00:08.257217+0000","last_clean":"2024-04-25T11:00:08.257217+0000","last_became_active":"2024-04-25T11:00:07.258222+0000","last_became_peered":"2024-04-25T11:00:07.258222+0000","last_unstale":"2024-04-25T11:00:08.257217+0000","last_undegraded":"2024-04-25T11:00:08.257217+0000","last_fullsized":"2024-04-25T11:00:08.257217+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:56.469604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"30'4","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265498+0000","last_change":"2024-04-25T11:00:06.257849+0000","last_active":"2024-04-25T11:00:08.265498+0000","last_peered":"2024-04-25T11:00:08.265498+0000","last_clean":"2024-04-25T11:00:08.265498+0000","last_became_active":"2024-04-25T11:00:06.257452+0000","last_became_peered":"2024-04-25T11:00:06.257452+0000","last_unstale":"2024-04-25T11:00:08.265498+0000","last_undegraded":"2024-04-25T11:00:08.265498+0000","last_fullsized":"2024-04-25T11:00:08.265498+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:37.048525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265371+0000","last_change":"2024-04-25T11:00:07.258826+0000","last_active":"2024-04-25T11:00:08.265371+0000","last_peered":"2024-04-25T11:00:08.265371+0000","last_clean":"2024-04-25T11:00:08.265371+0000","last_became_active":"2024-04-25T11:00:07.258379+0000","last_became_peered":"2024-04-25T11:00:07.258379+0000","last_unstale":"2024-04-25T11:00:08.265371+0000","last_undegraded":"2024-04-25T11:00:08.265371+0000","last_fullsized":"2024-04-25T11:00:08.265371+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:21.042782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.257184+0000","last_change":"2024-04-25T11:00:06.265699+0000","last_active":"2024-04-25T11:00:08.257184+0000","last_peered":"2024-04-25T11:00:08.257184+0000","last_clean":"2024-04-25T11:00:08.257184+0000","last_became_active":"2024-04-25T11:00:06.265112+0000","last_became_peered":"2024-04-25T11:00:06.265112+0000","last_unstale":"2024-04-25T11:00:08.257184+0000","last_undegraded":"2024-04-25T11:00:08.257184+0000","last_fullsized":"2024-04-25T11:00:08.257184+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:02.917078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265501+0000","last_change":"2024-04-25T11:00:07.265206+0000","last_active":"2024-04-25T11:00:08.265501+0000","last_peered":"2024-04-25T11:00:08.265501+0000","last_clean":"2024-04-25T11:00:08.265501+0000","last_became_active":"2024-04-25T11:00:07.264877+0000","last_became_peered":"2024-04-25T11:00:07.264877+0000","last_unstale":"2024-04-25T11:00:08.265501+0000","last_undegraded":"2024-04-25T11:00:08.265501+0000","last_fullsized":"2024-04-25T11:00:08.265501+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:20.203175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.834866+0000","last_change":"2024-04-25T11:00:06.262272+0000","last_active":"2024-04-25T11:00:08.834866+0000","last_peered":"2024-04-25T11:00:08.834866+0000","last_clean":"2024-04-25T11:00:08.834866+0000","last_became_active":"2024-04-25T11:00:06.261982+0000","last_became_peered":"2024-04-25T11:00:06.261982+0000","last_unstale":"2024-04-25T11:00:08.834866+0000","last_undegraded":"2024-04-25T11:00:08.834866+0000","last_fullsized":"2024-04-25T11:00:08.834866+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:47.597884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.260431+0000","last_change":"2024-04-25T11:00:07.272099+0000","last_active":"2024-04-25T11:00:08.260431+0000","last_peered":"2024-04-25T11:00:08.260431+0000","last_clean":"2024-04-25T11:00:08.260431+0000","last_became_active":"2024-04-25T11:00:07.271755+0000","last_became_peered":"2024-04-25T11:00:07.271755+0000","last_unstale":"2024-04-25T11:00:08.260431+0000","last_undegraded":"2024-04-25T11:00:08.260431+0000","last_fullsized":"2024-04-25T11:00:08.260431+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:11.896031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835040+0000","last_change":"2024-04-25T11:00:06.257693+0000","last_active":"2024-04-25T11:00:08.835040+0000","last_peered":"2024-04-25T11:00:08.835040+0000","last_clean":"2024-04-25T11:00:08.835040+0000","last_became_active":"2024-04-25T11:00:06.257402+0000","last_became_peered":"2024-04-25T11:00:06.257402+0000","last_unstale":"2024-04-25T11:00:08.835040+0000","last_undegraded":"2024-04-25T11:00:08.835040+0000","last_fullsized":"2024-04-25T11:00:08.835040+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:31.946740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835381+0000","last_change":"2024-04-25T11:00:07.270211+0000","last_active":"2024-04-25T11:00:08.835381+0000","last_peered":"2024-04-25T11:00:08.835381+0000","last_clean":"2024-04-25T11:00:08.835381+0000","last_became_active":"2024-04-25T11:00:07.269871+0000","last_became_peered":"2024-04-25T11:00:07.269871+0000","last_unstale":"2024-04-25T11:00:08.835381+0000","last_undegraded":"2024-04-25T11:00:08.835381+0000","last_fullsized":"2024-04-25T11:00:08.835381+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:02.017251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265828+0000","last_change":"2024-04-25T11:00:06.260758+0000","last_active":"2024-04-25T11:00:08.265828+0000","last_peered":"2024-04-25T11:00:08.265828+0000","last_clean":"2024-04-25T11:00:08.265828+0000","last_became_active":"2024-04-25T11:00:06.260455+0000","last_became_peered":"2024-04-25T11:00:06.260455+0000","last_unstale":"2024-04-25T11:00:08.265828+0000","last_undegraded":"2024-04-25T11:00:08.265828+0000","last_fullsized":"2024-04-25T11:00:08.265828+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:37.048525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.333932+0000","last_change":"2024-04-25T11:00:07.259280+0000","last_active":"2024-04-25T11:00:08.333932+0000","last_peered":"2024-04-25T11:00:08.333932+0000","last_clean":"2024-04-25T11:00:08.333932+0000","last_became_active":"2024-04-25T11:00:07.258640+0000","last_became_peered":"2024-04-25T11:00:07.258640+0000","last_unstale":"2024-04-25T11:00:08.333932+0000","last_undegraded":"2024-04-25T11:00:08.333932+0000","last_fullsized":"2024-04-25T11:00:08.333932+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:38.615241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265374+0000","last_change":"2024-04-25T11:00:06.255855+0000","last_active":"2024-04-25T11:00:08.265374+0000","last_peered":"2024-04-25T11:00:08.265374+0000","last_clean":"2024-04-25T11:00:08.265374+0000","last_became_active":"2024-04-25T11:00:06.255509+0000","last_became_peered":"2024-04-25T11:00:06.255509+0000","last_unstale":"2024-04-25T11:00:08.265374+0000","last_undegraded":"2024-04-25T11:00:08.265374+0000","last_fullsized":"2024-04-25T11:00:08.265374+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:55.930698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.264719+0000","last_change":"2024-04-25T11:00:07.253754+0000","last_active":"2024-04-25T11:00:08.264719+0000","last_peered":"2024-04-25T11:00:08.264719+0000","last_clean":"2024-04-25T11:00:08.264719+0000","last_became_active":"2024-04-25T11:00:07.253424+0000","last_became_peered":"2024-04-25T11:00:07.253424+0000","last_unstale":"2024-04-25T11:00:08.264719+0000","last_undegraded":"2024-04-25T11:00:08.264719+0000","last_fullsized":"2024-04-25T11:00:08.264719+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:17.734316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.834560+0000","last_change":"2024-04-25T11:00:06.259273+0000","last_active":"2024-04-25T11:00:08.834560+0000","last_peered":"2024-04-25T11:00:08.834560+0000","last_clean":"2024-04-25T11:00:08.834560+0000","last_became_active":"2024-04-25T11:00:06.258923+0000","last_became_peered":"2024-04-25T11:00:06.258923+0000","last_unstale":"2024-04-25T11:00:08.834560+0000","last_undegraded":"2024-04-25T11:00:08.834560+0000","last_fullsized":"2024-04-25T11:00:08.834560+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:20.035179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.259700+0000","last_change":"2024-04-25T11:00:07.266213+0000","last_active":"2024-04-25T11:00:08.259700+0000","last_peered":"2024-04-25T11:00:08.259700+0000","last_clean":"2024-04-25T11:00:08.259700+0000","last_became_active":"2024-04-25T11:00:07.265846+0000","last_became_peered":"2024-04-25T11:00:07.265846+0000","last_unstale":"2024-04-25T11:00:08.259700+0000","last_undegraded":"2024-04-25T11:00:08.259700+0000","last_fullsized":"2024-04-25T11:00:08.259700+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:51.748199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.264850+0000","last_change":"2024-04-25T11:00:06.259650+0000","last_active":"2024-04-25T11:00:08.264850+0000","last_peered":"2024-04-25T11:00:08.264850+0000","last_clean":"2024-04-25T11:00:08.264850+0000","last_became_active":"2024-04-25T11:00:06.259152+0000","last_became_peered":"2024-04-25T11:00:06.259152+0000","last_unstale":"2024-04-25T11:00:08.264850+0000","last_undegraded":"2024-04-25T11:00:08.264850+0000","last_fullsized":"2024-04-25T11:00:08.264850+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:30.099257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.334408+0000","last_change":"2024-04-25T11:00:07.265671+0000","last_active":"2024-04-25T11:00:08.334408+0000","last_peered":"2024-04-25T11:00:08.334408+0000","last_clean":"2024-04-25T11:00:08.334408+0000","last_became_active":"2024-04-25T11:00:07.265322+0000","last_became_peered":"2024-04-25T11:00:07.265322+0000","last_unstale":"2024-04-25T11:00:08.334408+0000","last_undegraded":"2024-04-25T11:00:08.334408+0000","last_fullsized":"2024-04-25T11:00:08.334408+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:02.017251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835351+0000","last_change":"2024-04-25T11:00:06.266165+0000","last_active":"2024-04-25T11:00:08.835351+0000","last_peered":"2024-04-25T11:00:08.835351+0000","last_clean":"2024-04-25T11:00:08.835351+0000","last_became_active":"2024-04-25T11:00:06.265773+0000","last_became_peered":"2024-04-25T11:00:06.265773+0000","last_unstale":"2024-04-25T11:00:08.835351+0000","last_undegraded":"2024-04-25T11:00:08.835351+0000","last_fullsized":"2024-04-25T11:00:08.835351+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:10.861006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835282+0000","last_change":"2024-04-25T11:00:07.272301+0000","last_active":"2024-04-25T11:00:08.835282+0000","last_peered":"2024-04-25T11:00:08.835282+0000","last_clean":"2024-04-25T11:00:08.835282+0000","last_became_active":"2024-04-25T11:00:07.271957+0000","last_became_peered":"2024-04-25T11:00:07.271957+0000","last_unstale":"2024-04-25T11:00:08.835282+0000","last_undegraded":"2024-04-25T11:00:08.835282+0000","last_fullsized":"2024-04-25T11:00:08.835282+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:03.486987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.257266+0000","last_change":"2024-04-25T11:00:06.259862+0000","last_active":"2024-04-25T11:00:08.257266+0000","last_peered":"2024-04-25T11:00:08.257266+0000","last_clean":"2024-04-25T11:00:08.257266+0000","last_became_active":"2024-04-25T11:00:06.259570+0000","last_became_peered":"2024-04-25T11:00:06.259570+0000","last_unstale":"2024-04-25T11:00:08.257266+0000","last_undegraded":"2024-04-25T11:00:08.257266+0000","last_fullsized":"2024-04-25T11:00:08.257266+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:47.597884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835894+0000","last_change":"2024-04-25T11:00:07.272966+0000","last_active":"2024-04-25T11:00:08.835894+0000","last_peered":"2024-04-25T11:00:08.835894+0000","last_clean":"2024-04-25T11:00:08.835894+0000","last_became_active":"2024-04-25T11:00:07.272568+0000","last_became_peered":"2024-04-25T11:00:07.272568+0000","last_unstale":"2024-04-25T11:00:08.835894+0000","last_undegraded":"2024-04-25T11:00:08.835894+0000","last_fullsized":"2024-04-25T11:00:08.835894+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:02.017251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"30'3","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.334266+0000","last_change":"2024-04-25T11:00:06.266271+0000","last_active":"2024-04-25T11:00:08.334266+0000","last_peered":"2024-04-25T11:00:08.334266+0000","last_clean":"2024-04-25T11:00:08.334266+0000","last_became_active":"2024-04-25T11:00:06.265815+0000","last_became_peered":"2024-04-25T11:00:06.265815+0000","last_unstale":"2024-04-25T11:00:08.334266+0000","last_undegraded":"2024-04-25T11:00:08.334266+0000","last_fullsized":"2024-04-25T11:00:08.334266+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:20.035179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.334126+0000","last_change":"2024-04-25T11:00:07.270387+0000","last_active":"2024-04-25T11:00:08.334126+0000","last_peered":"2024-04-25T11:00:08.334126+0000","last_clean":"2024-04-25T11:00:08.334126+0000","last_became_active":"2024-04-25T11:00:07.270095+0000","last_became_peered":"2024-04-25T11:00:07.270095+0000","last_unstale":"2024-04-25T11:00:08.334126+0000","last_undegraded":"2024-04-25T11:00:08.334126+0000","last_fullsized":"2024-04-25T11:00:08.334126+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:16.167953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"30'6","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835675+0000","last_change":"2024-04-25T11:00:06.266846+0000","last_active":"2024-04-25T11:00:08.835675+0000","last_peered":"2024-04-25T11:00:08.835675+0000","last_clean":"2024-04-25T11:00:08.835675+0000","last_became_active":"2024-04-25T11:00:06.266553+0000","last_became_peered":"2024-04-25T11:00:06.266553+0000","last_unstale":"2024-04-25T11:00:08.835675+0000","last_undegraded":"2024-04-25T11:00:08.835675+0000","last_fullsized":"2024-04-25T11:00:08.835675+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:59.507845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.259726+0000","last_change":"2024-04-25T11:00:07.268836+0000","last_active":"2024-04-25T11:00:08.259726+0000","last_peered":"2024-04-25T11:00:08.259726+0000","last_clean":"2024-04-25T11:00:08.259726+0000","last_became_active":"2024-04-25T11:00:07.268518+0000","last_became_peered":"2024-04-25T11:00:07.268518+0000","last_unstale":"2024-04-25T11:00:08.259726+0000","last_undegraded":"2024-04-25T11:00:08.259726+0000","last_fullsized":"2024-04-25T11:00:08.259726+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:56.469604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265928+0000","last_change":"2024-04-25T11:00:06.258144+0000","last_active":"2024-04-25T11:00:08.265928+0000","last_peered":"2024-04-25T11:00:08.265928+0000","last_clean":"2024-04-25T11:00:08.265928+0000","last_became_active":"2024-04-25T11:00:06.257775+0000","last_became_peered":"2024-04-25T11:00:06.257775+0000","last_unstale":"2024-04-25T11:00:08.265928+0000","last_undegraded":"2024-04-25T11:00:08.265928+0000","last_fullsized":"2024-04-25T11:00:08.265928+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:03.333540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835562+0000","last_change":"2024-04-25T11:00:07.270774+0000","last_active":"2024-04-25T11:00:08.835562+0000","last_peered":"2024-04-25T11:00:08.835562+0000","last_clean":"2024-04-25T11:00:08.835562+0000","last_became_active":"2024-04-25T11:00:07.270481+0000","last_became_peered":"2024-04-25T11:00:07.270481+0000","last_unstale":"2024-04-25T11:00:08.835562+0000","last_undegraded":"2024-04-25T11:00:08.835562+0000","last_fullsized":"2024-04-25T11:00:08.835562+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:16.167953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.259786+0000","last_change":"2024-04-25T11:00:06.260717+0000","last_active":"2024-04-25T11:00:08.259786+0000","last_peered":"2024-04-25T11:00:08.259786+0000","last_clean":"2024-04-25T11:00:08.259786+0000","last_became_active":"2024-04-25T11:00:06.260433+0000","last_became_peered":"2024-04-25T11:00:06.260433+0000","last_unstale":"2024-04-25T11:00:08.259786+0000","last_undegraded":"2024-04-25T11:00:08.259786+0000","last_fullsized":"2024-04-25T11:00:08.259786+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:02.917078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835298+0000","last_change":"2024-04-25T11:00:07.271908+0000","last_active":"2024-04-25T11:00:08.835298+0000","last_peered":"2024-04-25T11:00:08.835298+0000","last_clean":"2024-04-25T11:00:08.835298+0000","last_became_active":"2024-04-25T11:00:07.271482+0000","last_became_peered":"2024-04-25T11:00:07.271482+0000","last_unstale":"2024-04-25T11:00:08.835298+0000","last_undegraded":"2024-04-25T11:00:08.835298+0000","last_fullsized":"2024-04-25T11:00:08.835298+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:19.428784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.834707+0000","last_change":"2024-04-25T11:00:06.265403+0000","last_active":"2024-04-25T11:00:08.834707+0000","last_peered":"2024-04-25T11:00:08.834707+0000","last_clean":"2024-04-25T11:00:08.834707+0000","last_became_active":"2024-04-25T11:00:06.265062+0000","last_became_peered":"2024-04-25T11:00:06.265062+0000","last_unstale":"2024-04-25T11:00:08.834707+0000","last_undegraded":"2024-04-25T11:00:08.834707+0000","last_fullsized":"2024-04-25T11:00:08.834707+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:45.067082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265012+0000","last_change":"2024-04-25T11:00:07.260724+0000","last_active":"2024-04-25T11:00:08.265012+0000","last_peered":"2024-04-25T11:00:08.265012+0000","last_clean":"2024-04-25T11:00:08.265012+0000","last_became_active":"2024-04-25T11:00:07.259749+0000","last_became_peered":"2024-04-25T11:00:07.259749+0000","last_unstale":"2024-04-25T11:00:08.265012+0000","last_undegraded":"2024-04-25T11:00:08.265012+0000","last_fullsized":"2024-04-25T11:00:08.265012+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:28.820953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"30'8","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.834506+0000","last_change":"2024-04-25T11:00:06.266837+0000","last_active":"2024-04-25T11:00:08.834506+0000","last_peered":"2024-04-25T11:00:08.834506+0000","last_clean":"2024-04-25T11:00:08.834506+0000","last_became_active":"2024-04-25T11:00:06.266003+0000","last_became_peered":"2024-04-25T11:00:06.266003+0000","last_unstale":"2024-04-25T11:00:08.834506+0000","last_undegraded":"2024-04-25T11:00:08.834506+0000","last_fullsized":"2024-04-25T11:00:08.834506+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:23.511816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.264765+0000","last_change":"2024-04-25T11:00:07.261111+0000","last_active":"2024-04-25T11:00:08.264765+0000","last_peered":"2024-04-25T11:00:08.264765+0000","last_clean":"2024-04-25T11:00:08.264765+0000","last_became_active":"2024-04-25T11:00:07.260768+0000","last_became_peered":"2024-04-25T11:00:07.260768+0000","last_unstale":"2024-04-25T11:00:08.264765+0000","last_undegraded":"2024-04-25T11:00:08.264765+0000","last_fullsized":"2024-04-25T11:00:08.264765+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:00.515448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.260341+0000","last_change":"2024-04-25T11:00:06.262237+0000","last_active":"2024-04-25T11:00:08.260341+0000","last_peered":"2024-04-25T11:00:08.260341+0000","last_clean":"2024-04-25T11:00:08.260341+0000","last_became_active":"2024-04-25T11:00:06.261872+0000","last_became_peered":"2024-04-25T11:00:06.261872+0000","last_unstale":"2024-04-25T11:00:08.260341+0000","last_undegraded":"2024-04-25T11:00:08.260341+0000","last_fullsized":"2024-04-25T11:00:08.260341+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:03.333540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.334370+0000","last_change":"2024-04-25T11:00:07.264556+0000","last_active":"2024-04-25T11:00:08.334370+0000","last_peered":"2024-04-25T11:00:08.334370+0000","last_clean":"2024-04-25T11:00:08.334370+0000","last_became_active":"2024-04-25T11:00:07.263925+0000","last_became_peered":"2024-04-25T11:00:07.263925+0000","last_unstale":"2024-04-25T11:00:08.334370+0000","last_undegraded":"2024-04-25T11:00:08.334370+0000","last_fullsized":"2024-04-25T11:00:08.334370+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:19.428784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.264897+0000","last_change":"2024-04-25T11:00:06.257930+0000","last_active":"2024-04-25T11:00:08.264897+0000","last_peered":"2024-04-25T11:00:08.264897+0000","last_clean":"2024-04-25T11:00:08.264897+0000","last_became_active":"2024-04-25T11:00:06.257326+0000","last_became_peered":"2024-04-25T11:00:06.257326+0000","last_unstale":"2024-04-25T11:00:08.264897+0000","last_undegraded":"2024-04-25T11:00:08.264897+0000","last_fullsized":"2024-04-25T11:00:08.264897+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:11.002336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.259925+0000","last_change":"2024-04-25T11:00:07.261224+0000","last_active":"2024-04-25T11:00:08.259925+0000","last_peered":"2024-04-25T11:00:08.259925+0000","last_clean":"2024-04-25T11:00:08.259925+0000","last_became_active":"2024-04-25T11:00:07.260917+0000","last_became_peered":"2024-04-25T11:00:07.260917+0000","last_unstale":"2024-04-25T11:00:08.259925+0000","last_undegraded":"2024-04-25T11:00:08.259925+0000","last_fullsized":"2024-04-25T11:00:08.259925+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:50.570436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265046+0000","last_change":"2024-04-25T11:00:06.256448+0000","last_active":"2024-04-25T11:00:08.265046+0000","last_peered":"2024-04-25T11:00:08.265046+0000","last_clean":"2024-04-25T11:00:08.265046+0000","last_became_active":"2024-04-25T11:00:06.256095+0000","last_became_peered":"2024-04-25T11:00:06.256095+0000","last_unstale":"2024-04-25T11:00:08.265046+0000","last_undegraded":"2024-04-25T11:00:08.265046+0000","last_fullsized":"2024-04-25T11:00:08.265046+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:37.511626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835002+0000","last_change":"2024-04-25T11:00:07.263575+0000","last_active":"2024-04-25T11:00:08.835002+0000","last_peered":"2024-04-25T11:00:08.835002+0000","last_clean":"2024-04-25T11:00:08.835002+0000","last_became_active":"2024-04-25T11:00:07.263296+0000","last_became_peered":"2024-04-25T11:00:07.263296+0000","last_unstale":"2024-04-25T11:00:08.835002+0000","last_undegraded":"2024-04-25T11:00:08.835002+0000","last_fullsized":"2024-04-25T11:00:08.835002+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:24.519419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265630+0000","last_change":"2024-04-25T11:00:06.261291+0000","last_active":"2024-04-25T11:00:08.265630+0000","last_peered":"2024-04-25T11:00:08.265630+0000","last_clean":"2024-04-25T11:00:08.265630+0000","last_became_active":"2024-04-25T11:00:06.260969+0000","last_became_peered":"2024-04-25T11:00:06.260969+0000","last_unstale":"2024-04-25T11:00:08.265630+0000","last_undegraded":"2024-04-25T11:00:08.265630+0000","last_fullsized":"2024-04-25T11:00:08.265630+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:08.488342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.834714+0000","last_change":"2024-04-25T11:00:07.258332+0000","last_active":"2024-04-25T11:00:08.834714+0000","last_peered":"2024-04-25T11:00:08.834714+0000","last_clean":"2024-04-25T11:00:08.834714+0000","last_became_active":"2024-04-25T11:00:07.258010+0000","last_became_peered":"2024-04-25T11:00:07.258010+0000","last_unstale":"2024-04-25T11:00:08.834714+0000","last_undegraded":"2024-04-25T11:00:08.834714+0000","last_fullsized":"2024-04-25T11:00:08.834714+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:03.924681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.257049+0000","last_change":"2024-04-25T11:00:06.259265+0000","last_active":"2024-04-25T11:00:08.257049+0000","last_peered":"2024-04-25T11:00:08.257049+0000","last_clean":"2024-04-25T11:00:08.257049+0000","last_became_active":"2024-04-25T11:00:06.258949+0000","last_became_peered":"2024-04-25T11:00:06.258949+0000","last_unstale":"2024-04-25T11:00:08.257049+0000","last_undegraded":"2024-04-25T11:00:08.257049+0000","last_fullsized":"2024-04-25T11:00:08.257049+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:55.930698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.834983+0000","last_change":"2024-04-25T11:00:07.258792+0000","last_active":"2024-04-25T11:00:08.834983+0000","last_peered":"2024-04-25T11:00:08.834983+0000","last_clean":"2024-04-25T11:00:08.834983+0000","last_became_active":"2024-04-25T11:00:07.258502+0000","last_became_peered":"2024-04-25T11:00:07.258502+0000","last_unstale":"2024-04-25T11:00:08.834983+0000","last_undegraded":"2024-04-25T11:00:08.834983+0000","last_fullsized":"2024-04-25T11:00:08.834983+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:11.868608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.264585+0000","last_change":"2024-04-25T11:00:06.255869+0000","last_active":"2024-04-25T11:00:08.264585+0000","last_peered":"2024-04-25T11:00:08.264585+0000","last_clean":"2024-04-25T11:00:08.264585+0000","last_became_active":"2024-04-25T11:00:06.255468+0000","last_became_peered":"2024-04-25T11:00:06.255468+0000","last_unstale":"2024-04-25T11:00:08.264585+0000","last_undegraded":"2024-04-25T11:00:08.264585+0000","last_fullsized":"2024-04-25T11:00:08.264585+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:08.488342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.834579+0000","last_change":"2024-04-25T11:00:07.259018+0000","last_active":"2024-04-25T11:00:08.834579+0000","last_peered":"2024-04-25T11:00:08.834579+0000","last_clean":"2024-04-25T11:00:08.834579+0000","last_became_active":"2024-04-25T11:00:07.258712+0000","last_became_peered":"2024-04-25T11:00:07.258712+0000","last_unstale":"2024-04-25T11:00:08.834579+0000","last_undegraded":"2024-04-25T11:00:08.834579+0000","last_fullsized":"2024-04-25T11:00:08.834579+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:04.341143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265489+0000","last_change":"2024-04-25T11:00:06.259165+0000","last_active":"2024-04-25T11:00:08.265489+0000","last_peered":"2024-04-25T11:00:08.265489+0000","last_clean":"2024-04-25T11:00:08.265489+0000","last_became_active":"2024-04-25T11:00:06.258814+0000","last_became_peered":"2024-04-25T11:00:06.258814+0000","last_unstale":"2024-04-25T11:00:08.265489+0000","last_undegraded":"2024-04-25T11:00:08.265489+0000","last_fullsized":"2024-04-25T11:00:08.265489+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:40.478099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.834973+0000","last_change":"2024-04-25T11:00:07.262688+0000","last_active":"2024-04-25T11:00:08.834973+0000","last_peered":"2024-04-25T11:00:08.834973+0000","last_clean":"2024-04-25T11:00:08.834973+0000","last_became_active":"2024-04-25T11:00:07.262213+0000","last_became_peered":"2024-04-25T11:00:07.262213+0000","last_unstale":"2024-04-25T11:00:08.834973+0000","last_undegraded":"2024-04-25T11:00:08.834973+0000","last_fullsized":"2024-04-25T11:00:08.834973+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:51.247079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.333802+0000","last_change":"2024-04-25T11:00:06.266180+0000","last_active":"2024-04-25T11:00:08.333802+0000","last_peered":"2024-04-25T11:00:08.333802+0000","last_clean":"2024-04-25T11:00:08.333802+0000","last_became_active":"2024-04-25T11:00:06.265801+0000","last_became_peered":"2024-04-25T11:00:06.265801+0000","last_unstale":"2024-04-25T11:00:08.333802+0000","last_undegraded":"2024-04-25T11:00:08.333802+0000","last_fullsized":"2024-04-25T11:00:08.333802+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:16.726713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.834713+0000","last_change":"2024-04-25T11:00:07.260694+0000","last_active":"2024-04-25T11:00:08.834713+0000","last_peered":"2024-04-25T11:00:08.834713+0000","last_clean":"2024-04-25T11:00:08.834713+0000","last_became_active":"2024-04-25T11:00:07.260414+0000","last_became_peered":"2024-04-25T11:00:07.260414+0000","last_unstale":"2024-04-25T11:00:08.834713+0000","last_undegraded":"2024-04-25T11:00:08.834713+0000","last_fullsized":"2024-04-25T11:00:08.834713+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:43.915729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.260364+0000","last_change":"2024-04-25T11:00:06.257887+0000","last_active":"2024-04-25T11:00:08.260364+0000","last_peered":"2024-04-25T11:00:08.260364+0000","last_clean":"2024-04-25T11:00:08.260364+0000","last_became_active":"2024-04-25T11:00:06.257563+0000","last_became_peered":"2024-04-25T11:00:06.257563+0000","last_unstale":"2024-04-25T11:00:08.260364+0000","last_undegraded":"2024-04-25T11:00:08.260364+0000","last_fullsized":"2024-04-25T11:00:08.260364+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:55.930698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.334478+0000","last_change":"2024-04-25T11:00:07.271840+0000","last_active":"2024-04-25T11:00:08.334478+0000","last_peered":"2024-04-25T11:00:08.334478+0000","last_clean":"2024-04-25T11:00:08.334478+0000","last_became_active":"2024-04-25T11:00:07.269612+0000","last_became_peered":"2024-04-25T11:00:07.269612+0000","last_unstale":"2024-04-25T11:00:08.334478+0000","last_undegraded":"2024-04-25T11:00:08.334478+0000","last_fullsized":"2024-04-25T11:00:08.334478+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:07.367840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"31'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.257365+0000","last_change":"2024-04-25T11:00:06.259220+0000","last_active":"2024-04-25T11:00:08.257365+0000","last_peered":"2024-04-25T11:00:08.257365+0000","last_clean":"2024-04-25T11:00:08.257365+0000","last_became_active":"2024-04-25T11:00:06.258895+0000","last_became_peered":"2024-04-25T11:00:06.258895+0000","last_unstale":"2024-04-25T11:00:08.257365+0000","last_undegraded":"2024-04-25T11:00:08.257365+0000","last_fullsized":"2024-04-25T11:00:08.257365+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:10.861006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.256984+0000","last_change":"2024-04-25T11:00:07.271940+0000","last_active":"2024-04-25T11:00:08.256984+0000","last_peered":"2024-04-25T11:00:08.256984+0000","last_clean":"2024-04-25T11:00:08.256984+0000","last_became_active":"2024-04-25T11:00:07.271640+0000","last_became_peered":"2024-04-25T11:00:07.271640+0000","last_unstale":"2024-04-25T11:00:08.256984+0000","last_undegraded":"2024-04-25T11:00:08.256984+0000","last_fullsized":"2024-04-25T11:00:08.256984+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:00.515448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.259999+0000","last_change":"2024-04-25T11:00:06.257498+0000","last_active":"2024-04-25T11:00:08.259999+0000","last_peered":"2024-04-25T11:00:08.259999+0000","last_clean":"2024-04-25T11:00:08.259999+0000","last_became_active":"2024-04-25T11:00:06.257083+0000","last_became_peered":"2024-04-25T11:00:06.257083+0000","last_unstale":"2024-04-25T11:00:08.259999+0000","last_undegraded":"2024-04-25T11:00:08.259999+0000","last_fullsized":"2024-04-25T11:00:08.259999+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:37.048525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.264607+0000","last_change":"2024-04-25T11:00:07.254833+0000","last_active":"2024-04-25T11:00:08.264607+0000","last_peered":"2024-04-25T11:00:08.264607+0000","last_clean":"2024-04-25T11:00:08.264607+0000","last_became_active":"2024-04-25T11:00:07.254472+0000","last_became_peered":"2024-04-25T11:00:07.254472+0000","last_unstale":"2024-04-25T11:00:08.264607+0000","last_undegraded":"2024-04-25T11:00:08.264607+0000","last_fullsized":"2024-04-25T11:00:08.264607+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:11.868608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.257207+0000","last_change":"2024-04-25T11:00:06.257789+0000","last_active":"2024-04-25T11:00:08.257207+0000","last_peered":"2024-04-25T11:00:08.257207+0000","last_clean":"2024-04-25T11:00:08.257207+0000","last_became_active":"2024-04-25T11:00:06.257450+0000","last_became_peered":"2024-04-25T11:00:06.257450+0000","last_unstale":"2024-04-25T11:00:08.257207+0000","last_undegraded":"2024-04-25T11:00:08.257207+0000","last_fullsized":"2024-04-25T11:00:08.257207+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:42.908127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.257067+0000","last_change":"2024-04-25T11:00:07.255875+0000","last_active":"2024-04-25T11:00:08.257067+0000","last_peered":"2024-04-25T11:00:08.257067+0000","last_clean":"2024-04-25T11:00:08.257067+0000","last_became_active":"2024-04-25T11:00:07.255565+0000","last_became_peered":"2024-04-25T11:00:07.255565+0000","last_unstale":"2024-04-25T11:00:08.257067+0000","last_undegraded":"2024-04-25T11:00:08.257067+0000","last_fullsized":"2024-04-25T11:00:08.257067+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:51.748199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.259845+0000","last_change":"2024-04-25T11:00:06.257884+0000","last_active":"2024-04-25T11:00:08.259845+0000","last_peered":"2024-04-25T11:00:08.259845+0000","last_clean":"2024-04-25T11:00:08.259845+0000","last_became_active":"2024-04-25T11:00:06.257586+0000","last_became_peered":"2024-04-25T11:00:06.257586+0000","last_unstale":"2024-04-25T11:00:08.259845+0000","last_undegraded":"2024-04-25T11:00:08.259845+0000","last_fullsized":"2024-04-25T11:00:08.259845+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:42.908127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.834496+0000","last_change":"2024-04-25T11:00:07.274001+0000","last_active":"2024-04-25T11:00:08.834496+0000","last_peered":"2024-04-25T11:00:08.834496+0000","last_clean":"2024-04-25T11:00:08.834496+0000","last_became_active":"2024-04-25T11:00:07.273698+0000","last_became_peered":"2024-04-25T11:00:07.273698+0000","last_unstale":"2024-04-25T11:00:08.834496+0000","last_undegraded":"2024-04-25T11:00:08.834496+0000","last_fullsized":"2024-04-25T11:00:08.834496+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:18.962985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265502+0000","last_change":"2024-04-25T11:00:06.258316+0000","last_active":"2024-04-25T11:00:08.265502+0000","last_peered":"2024-04-25T11:00:08.265502+0000","last_clean":"2024-04-25T11:00:08.265502+0000","last_became_active":"2024-04-25T11:00:06.257838+0000","last_became_peered":"2024-04-25T11:00:06.257838+0000","last_unstale":"2024-04-25T11:00:08.265502+0000","last_undegraded":"2024-04-25T11:00:08.265502+0000","last_fullsized":"2024-04-25T11:00:08.265502+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:44.176669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.265356+0000","last_change":"2024-04-25T11:00:07.268312+0000","last_active":"2024-04-25T11:00:08.265356+0000","last_peered":"2024-04-25T11:00:08.265356+0000","last_clean":"2024-04-25T11:00:08.265356+0000","last_became_active":"2024-04-25T11:00:07.267952+0000","last_became_peered":"2024-04-25T11:00:07.267952+0000","last_unstale":"2024-04-25T11:00:08.265356+0000","last_undegraded":"2024-04-25T11:00:08.265356+0000","last_fullsized":"2024-04-25T11:00:08.265356+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:51.748199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.834608+0000","last_change":"2024-04-25T11:00:06.266105+0000","last_active":"2024-04-25T11:00:08.834608+0000","last_peered":"2024-04-25T11:00:08.834608+0000","last_clean":"2024-04-25T11:00:08.834608+0000","last_became_active":"2024-04-25T11:00:06.265786+0000","last_became_peered":"2024-04-25T11:00:06.265786+0000","last_unstale":"2024-04-25T11:00:08.834608+0000","last_undegraded":"2024-04-25T11:00:08.834608+0000","last_fullsized":"2024-04-25T11:00:08.834608+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:27.813350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835213+0000","last_change":"2024-04-25T11:00:07.265240+0000","last_active":"2024-04-25T11:00:08.835213+0000","last_peered":"2024-04-25T11:00:08.835213+0000","last_clean":"2024-04-25T11:00:08.835213+0000","last_became_active":"2024-04-25T11:00:07.264915+0000","last_became_peered":"2024-04-25T11:00:07.264915+0000","last_unstale":"2024-04-25T11:00:08.835213+0000","last_undegraded":"2024-04-25T11:00:08.835213+0000","last_fullsized":"2024-04-25T11:00:08.835213+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:06.235114+0000","last_clean_scrub_stamp":"2024-04-25T11:00:06.235114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:16.167953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.264616+0000","last_change":"2024-04-25T11:00:06.250003+0000","last_active":"2024-04-25T11:00:08.264616+0000","last_peered":"2024-04-25T11:00:08.264616+0000","last_clean":"2024-04-25T11:00:08.264616+0000","last_became_active":"2024-04-25T11:00:06.249632+0000","last_became_peered":"2024-04-25T11:00:06.249632+0000","last_unstale":"2024-04-25T11:00:08.264616+0000","last_undegraded":"2024-04-25T11:00:08.264616+0000","last_fullsized":"2024-04-25T11:00:08.264616+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:00:05.227511+0000","last_clean_scrub_stamp":"2024-04-25T11:00:05.227511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:11.002336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":129,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T11:00:08.835561+0000","last_change":"2024-04-25T10:59:07.839466+0000","last_active":"2024-04-25T11:00:08.835561+0000","last_peered":"2024-04-25T11:00:08.835561+0000","last_clean":"2024-04-25T11:00:08.835561+0000","last_became_active":"2024-04-25T10:59:07.836269+0000","last_became_peered":"2024-04-25T10:59:07.836269+0000","last_unstale":"2024-04-25T11:00:08.835561+0000","last_undegraded":"2024-04-25T11:00:08.835561+0000","last_fullsized":"2024-04-25T11:00:08.835561+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:06.549663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:06.549663+0000","last_clean_scrub_stamp":"2024-04-25T10:59:06.549663+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:21.575330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":946176,"data_stored":724908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":356352,"data_stored":135066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":356352,"data_stored":135122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":356352,"data_stored":134554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":364544,"data_stored":136084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":352256,"data_stored":134520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":364544,"data_stored":136084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":942080,"data_stored":724818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:00:16.029 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T11:00:16.030 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T11:00:16.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.180+0000 7f1008314640 1 Processor -- start 2024-04-25T11:00:16.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.180+0000 7f1008314640 1 -- start start 2024-04-25T11:00:16.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.181+0000 7f1008314640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f100006dce0 0x7f100015f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:16.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.181+0000 7f1008314640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f10000b0af0 0x7f100015faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:16.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.181+0000 7f1008314640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10000cdea0 con 0x7f10000cd9a0 2024-04-25T11:00:16.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.181+0000 7f1008314640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f10000ceab0 con 0x7f10000b0af0 2024-04-25T11:00:16.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.181+0000 7f1008314640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f10000ce180 con 0x7f100006dce0 2024-04-25T11:00:16.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.182+0000 7f10068b1640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f10000b0af0 0x7f100015faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:16.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.182+0000 7f10068b1640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f10000b0af0 0x7f100015faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51014/0 (socket says 172.21.15.69:51014) 2024-04-25T11:00:16.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.182+0000 7f10068b1640 1 -- 172.21.15.69:0/3312998237 learned_addr learned my addr 172.21.15.69:0/3312998237 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:16.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.182+0000 7f10058af640 1 --2- 172.21.15.69:0/3312998237 >> v2:172.21.15.146:3300/0 conn(0x7f100006dce0 0x7f100015f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:16.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.182+0000 7f10068b1640 1 -- 172.21.15.69:0/3312998237 >> v2:172.21.15.146:3300/0 conn(0x7f100006dce0 msgr2=0x7f100015f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:16.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.182+0000 7f10068b1640 1 --2- 172.21.15.69:0/3312998237 >> v2:172.21.15.146:3300/0 conn(0x7f100006dce0 0x7f100015f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:16.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.182+0000 7f10068b1640 1 -- 172.21.15.69:0/3312998237 >> v1:172.21.15.146:6789/0 conn(0x7f10000cd9a0 legacy=0x7f100006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:16.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.182+0000 7f10068b1640 1 -- 172.21.15.69:0/3312998237 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10000ce460 con 0x7f10000b0af0 2024-04-25T11:00:16.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.183+0000 7f10068b1640 1 --2- 172.21.15.69:0/3312998237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10000b0af0 0x7f100015faa0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1000162f90 tx=0x7f0ff802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4250e6449163bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:16.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.183+0000 7f10050ae640 1 -- 172.21.15.69:0/3312998237 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8036070 con 0x7f10000b0af0 2024-04-25T11:00:16.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.183+0000 7f10050ae640 1 -- 172.21.15.69:0/3312998237 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ff8031600 con 0x7f10000b0af0 2024-04-25T11:00:16.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.183+0000 7f10050ae640 1 -- 172.21.15.69:0/3312998237 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff80319d0 con 0x7f10000b0af0 2024-04-25T11:00:16.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.183+0000 7f1008314640 1 -- 172.21.15.69:0/3312998237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10000b0af0 msgr2=0x7f100015faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:16.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.183+0000 7f1008314640 1 --2- 172.21.15.69:0/3312998237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10000b0af0 0x7f100015faa0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1000162f90 tx=0x7f0ff802a730 comp rx=0 tx=0).stop 2024-04-25T11:00:16.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.184+0000 7f1008314640 1 -- 172.21.15.69:0/3312998237 shutdown_connections 2024-04-25T11:00:16.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.184+0000 7f1008314640 1 --2- 172.21.15.69:0/3312998237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10000b0af0 0x7f100015faa0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:16.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.184+0000 7f1008314640 1 --2- 172.21.15.69:0/3312998237 >> v2:172.21.15.146:3300/0 conn(0x7f100006dce0 0x7f100015f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:16.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.184+0000 7f1008314640 1 -- 172.21.15.69:0/3312998237 >> 172.21.15.69:0/3312998237 conn(0x7f1000073090 msgr2=0x7f10000734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:16.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.184+0000 7f1008314640 1 -- 172.21.15.69:0/3312998237 shutdown_connections 2024-04-25T11:00:16.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.184+0000 7f1008314640 1 -- 172.21.15.69:0/3312998237 wait complete. 2024-04-25T11:00:16.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.185+0000 7f1008314640 1 Processor -- start 2024-04-25T11:00:16.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.185+0000 7f1008314640 1 -- start start 2024-04-25T11:00:16.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.187+0000 7f1008314640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10000b0af0 0x7f1000140950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:16.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.187+0000 7f1008314640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f10000cd9a0 0x7f1000143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:16.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.187+0000 7f1008314640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1000143a50 0x7f1000158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:16.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.187+0000 7f1008314640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f10000cef40 con 0x7f10000b0af0 2024-04-25T11:00:16.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.187+0000 7f1008314640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f10000b0f00 con 0x7f1000143a50 2024-04-25T11:00:16.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.187+0000 7f1008314640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f10000cf0c0 con 0x7f10000cd9a0 2024-04-25T11:00:16.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.187+0000 7f10060b0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10000b0af0 0x7f1000140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:16.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.187+0000 7f10060b0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10000b0af0 0x7f1000140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51026/0 (socket says 172.21.15.69:51026) 2024-04-25T11:00:16.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.187+0000 7f10060b0640 1 -- 172.21.15.69:0/71180686 learned_addr learned my addr 172.21.15.69:0/71180686 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:16.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.187+0000 7f10058af640 1 --2- 172.21.15.69:0/71180686 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f10000cd9a0 0x7f1000143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:16.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.187+0000 7f10068b1640 1 --2- 172.21.15.69:0/71180686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1000143a50 0x7f1000158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:16.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.187+0000 7f10060b0640 1 -- 172.21.15.69:0/71180686 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f10000cd9a0 msgr2=0x7f1000143510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:16.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.187+0000 7f10060b0640 1 --2- 172.21.15.69:0/71180686 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f10000cd9a0 0x7f1000143510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:16.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.187+0000 7f10060b0640 1 -- 172.21.15.69:0/71180686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1000143a50 msgr2=0x7f1000158130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:16.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.187+0000 7f10060b0640 1 --2- 172.21.15.69:0/71180686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1000143a50 0x7f1000158130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:16.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.187+0000 7f10060b0640 1 -- 172.21.15.69:0/71180686 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f100015fd20 con 0x7f10000b0af0 2024-04-25T11:00:16.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.187+0000 7f10058af640 1 --2- 172.21.15.69:0/71180686 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f10000cd9a0 0x7f1000143510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:16.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.187+0000 7f10060b0640 1 --2- 172.21.15.69:0/71180686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10000b0af0 0x7f1000140950 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00c3c0 tx=0x7f0ffc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:16.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.187+0000 7f0ff6ffd640 1 -- 172.21.15.69:0/71180686 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc0052d0 con 0x7f10000b0af0 2024-04-25T11:00:16.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.187+0000 7f0ff6ffd640 1 -- 172.21.15.69:0/71180686 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ffc005490 con 0x7f10000b0af0 2024-04-25T11:00:16.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.187+0000 7f1008314640 1 -- 172.21.15.69:0/71180686 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1000141180 con 0x7f10000b0af0 2024-04-25T11:00:16.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.188+0000 7f0ff6ffd640 1 -- 172.21.15.69:0/71180686 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc007260 con 0x7f10000b0af0 2024-04-25T11:00:16.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.188+0000 7f1008314640 1 -- 172.21.15.69:0/71180686 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10001602c0 con 0x7f10000b0af0 2024-04-25T11:00:16.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.188+0000 7f1008314640 1 -- 172.21.15.69:0/71180686 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1000050740 con 0x7f10000b0af0 2024-04-25T11:00:16.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.189+0000 7f0ff6ffd640 1 -- 172.21.15.69:0/71180686 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0ffc007400 con 0x7f10000b0af0 2024-04-25T11:00:16.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.190+0000 7f0ff6ffd640 1 --2- 172.21.15.69:0/71180686 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0fe40744b0 0x7f0fe40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:16.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.190+0000 7f0ff6ffd640 1 -- 172.21.15.69:0/71180686 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f0ffc0946f0 con 0x7f10000b0af0 2024-04-25T11:00:16.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.190+0000 7f10058af640 1 --2- 172.21.15.69:0/71180686 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0fe40744b0 0x7f0fe40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:16.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.190+0000 7f10058af640 1 --2- 172.21.15.69:0/71180686 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0fe40744b0 0x7f0fe40769c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f10001446a0 tx=0x7f0ff0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:16.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.194+0000 7f0ff6ffd640 1 -- 172.21.15.69:0/71180686 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0ffc05ee80 con 0x7f10000b0af0 2024-04-25T11:00:16.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.439+0000 7f1008314640 1 -- 172.21.15.69:0/71180686 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f10000b4b40 con 0x7f10000b0af0 2024-04-25T11:00:16.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.441+0000 7f0ff6ffd640 1 -- 172.21.15.69:0/71180686 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7f0ffc05e780 con 0x7f10000b0af0 2024-04-25T11:00:16.442 INFO:teuthology.orchestra.run.smithi069.stdout:[client.0] 2024-04-25T11:00:16.442 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQBMNypmtv/zNxAAz2JcqUNyHu2ipkrbl8ONng== 2024-04-25T11:00:16.442 INFO:teuthology.orchestra.run.smithi069.stdout: caps mds = "allow" 2024-04-25T11:00:16.442 INFO:teuthology.orchestra.run.smithi069.stdout: caps mon = "allow r" 2024-04-25T11:00:16.443 INFO:teuthology.orchestra.run.smithi069.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T11:00:16.443 INFO:teuthology.orchestra.run.smithi069.stderr:updated caps for client.0 2024-04-25T11:00:16.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.444+0000 7f1008314640 1 -- 172.21.15.69:0/71180686 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0fe40744b0 msgr2=0x7f0fe40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:16.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.444+0000 7f1008314640 1 --2- 172.21.15.69:0/71180686 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0fe40744b0 0x7f0fe40769c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f10001446a0 tx=0x7f0ff0008040 comp rx=0 tx=0).stop 2024-04-25T11:00:16.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.444+0000 7f1008314640 1 -- 172.21.15.69:0/71180686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10000b0af0 msgr2=0x7f1000140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:16.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.444+0000 7f1008314640 1 --2- 172.21.15.69:0/71180686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10000b0af0 0x7f1000140950 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00c3c0 tx=0x7f0ffc00c880 comp rx=0 tx=0).stop 2024-04-25T11:00:16.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.444+0000 7f1008314640 1 -- 172.21.15.69:0/71180686 shutdown_connections 2024-04-25T11:00:16.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.444+0000 7f1008314640 1 --2- 172.21.15.69:0/71180686 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f0fe40744b0 0x7f0fe40769c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:16.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.444+0000 7f1008314640 1 --2- 172.21.15.69:0/71180686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1000143a50 0x7f1000158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:16.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.444+0000 7f1008314640 1 --2- 172.21.15.69:0/71180686 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f10000cd9a0 0x7f1000143510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:16.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.444+0000 7f1008314640 1 --2- 172.21.15.69:0/71180686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10000b0af0 0x7f1000140950 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:16.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.444+0000 7f1008314640 1 -- 172.21.15.69:0/71180686 >> 172.21.15.69:0/71180686 conn(0x7f1000073090 msgr2=0x7f100006d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:16.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.444+0000 7f1008314640 1 -- 172.21.15.69:0/71180686 shutdown_connections 2024-04-25T11:00:16.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.444+0000 7f1008314640 1 -- 172.21.15.69:0/71180686 wait complete. 2024-04-25T11:00:16.455 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:00:16.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.562+0000 7f11dfa9f640 1 Processor -- start 2024-04-25T11:00:16.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.562+0000 7f11dfa9f640 1 -- start start 2024-04-25T11:00:16.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.563+0000 7f11dfa9f640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f11d8142730 0x7f11d8142b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:16.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.564+0000 7f11dfa9f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11d8070270 0x7f11d8070680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:16.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.564+0000 7f11dfa9f640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11d8070bc0 con 0x7f11d81400a0 2024-04-25T11:00:16.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.564+0000 7f11dfa9f640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f11d81431d0 con 0x7f11d8142730 2024-04-25T11:00:16.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.564+0000 7f11dfa9f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f11d81434b0 con 0x7f11d8070270 2024-04-25T11:00:16.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.564+0000 7f11de03c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11d8070270 0x7f11d8070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:16.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.564+0000 7f11dd83b640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f11d81400a0 0x7f11d80b6890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.69:53978/0 (socket says 172.21.15.69:53978) 2024-04-25T11:00:16.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.564+0000 7f11dd03a640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f11d8142730 0x7f11d8142b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:16.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.564+0000 7f11dd83b640 1 -- 172.21.15.69:0/2261036092 learned_addr learned my addr 172.21.15.69:0/2261036092 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:16.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.564+0000 7f11dc839640 1 -- 172.21.15.69:0/2261036092 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 647731250 0 0) 0x7f11d4001490 con 0x7f11d81400a0 2024-04-25T11:00:16.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.564+0000 7f11dc839640 1 -- 172.21.15.69:0/2261036092 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11c4003520 con 0x7f11d81400a0 2024-04-25T11:00:16.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.564+0000 7f11de03c640 1 -- 172.21.15.69:0/2261036092 >> v2:172.21.15.146:3300/0 conn(0x7f11d8142730 msgr2=0x7f11d8142b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:16.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.564+0000 7f11de03c640 1 --2- 172.21.15.69:0/2261036092 >> v2:172.21.15.146:3300/0 conn(0x7f11d8142730 0x7f11d8142b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:16.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.564+0000 7f11de03c640 1 -- 172.21.15.69:0/2261036092 >> v1:172.21.15.146:6789/0 conn(0x7f11d81400a0 legacy=0x7f11d80b6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:16.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.564+0000 7f11de03c640 1 -- 172.21.15.69:0/2261036092 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d80b6090 con 0x7f11d8070270 2024-04-25T11:00:16.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.564+0000 7f11dd03a640 1 --2- 172.21.15.69:0/2261036092 >> v2:172.21.15.146:3300/0 conn(0x7f11d8142730 0x7f11d8142b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:00:16.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.565+0000 7f11de03c640 1 --2- 172.21.15.69:0/2261036092 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11d8070270 0x7f11d8070680 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f11d00097a0 tx=0x7f11d002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f16976afc03c28a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:16.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.565+0000 7f11dc839640 1 -- 172.21.15.69:0/2261036092 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0037070 con 0x7f11d8070270 2024-04-25T11:00:16.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.565+0000 7f11dc839640 1 -- 172.21.15.69:0/2261036092 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11d0032440 con 0x7f11d8070270 2024-04-25T11:00:16.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.565+0000 7f11dc839640 1 -- 172.21.15.69:0/2261036092 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d00327f0 con 0x7f11d8070270 2024-04-25T11:00:16.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.566+0000 7f11dfa9f640 1 -- 172.21.15.69:0/2261036092 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11d8070270 msgr2=0x7f11d8070680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:16.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.566+0000 7f11dfa9f640 1 --2- 172.21.15.69:0/2261036092 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11d8070270 0x7f11d8070680 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f11d00097a0 tx=0x7f11d002a180 comp rx=0 tx=0).stop 2024-04-25T11:00:16.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.566+0000 7f11dfa9f640 1 -- 172.21.15.69:0/2261036092 shutdown_connections 2024-04-25T11:00:16.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.566+0000 7f11dfa9f640 1 --2- 172.21.15.69:0/2261036092 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11d8070270 0x7f11d8070680 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:16.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.566+0000 7f11dfa9f640 1 --2- 172.21.15.69:0/2261036092 >> v2:172.21.15.146:3300/0 conn(0x7f11d8142730 0x7f11d8142b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:16.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.566+0000 7f11dfa9f640 1 -- 172.21.15.69:0/2261036092 >> 172.21.15.69:0/2261036092 conn(0x7f11d8073050 msgr2=0x7f11d8073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:16.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.566+0000 7f11dfa9f640 1 -- 172.21.15.69:0/2261036092 shutdown_connections 2024-04-25T11:00:16.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.566+0000 7f11dfa9f640 1 -- 172.21.15.69:0/2261036092 wait complete. 2024-04-25T11:00:16.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.568+0000 7f11dfa9f640 1 Processor -- start 2024-04-25T11:00:16.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.568+0000 7f11dfa9f640 1 -- start start 2024-04-25T11:00:16.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.569+0000 7f11dfa9f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11d8070270 0x7f11d813c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:16.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.569+0000 7f11dfa9f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11d81400a0 0x7f11d80ca310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:16.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.569+0000 7f11dfa9f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11d8142730 0x7f11d806f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:16.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.569+0000 7f11dfa9f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f11d80b49a0 con 0x7f11d81400a0 2024-04-25T11:00:16.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.569+0000 7f11dfa9f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f11d80cddb0 con 0x7f11d8070270 2024-04-25T11:00:16.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.569+0000 7f11dfa9f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f11d80b4b20 con 0x7f11d8142730 2024-04-25T11:00:16.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.569+0000 7f11dd03a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11d81400a0 0x7f11d80ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:16.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.569+0000 7f11dd03a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11d81400a0 0x7f11d80ca310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51032/0 (socket says 172.21.15.69:51032) 2024-04-25T11:00:16.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.569+0000 7f11dd03a640 1 -- 172.21.15.69:0/1556486241 learned_addr learned my addr 172.21.15.69:0/1556486241 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:16.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.569+0000 7f11de03c640 1 --2- 172.21.15.69:0/1556486241 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11d8142730 0x7f11d806f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:16.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.569+0000 7f11dd83b640 1 --2- 172.21.15.69:0/1556486241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11d8070270 0x7f11d813c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:16.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.570+0000 7f11dd03a640 1 -- 172.21.15.69:0/1556486241 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11d8142730 msgr2=0x7f11d806f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:16.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.570+0000 7f11dd03a640 1 --2- 172.21.15.69:0/1556486241 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11d8142730 0x7f11d806f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:16.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.570+0000 7f11dd03a640 1 -- 172.21.15.69:0/1556486241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11d8070270 msgr2=0x7f11d813c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:16.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.570+0000 7f11dd03a640 1 --2- 172.21.15.69:0/1556486241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11d8070270 0x7f11d813c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:16.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.570+0000 7f11dd03a640 1 -- 172.21.15.69:0/1556486241 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d80cd790 con 0x7f11d81400a0 2024-04-25T11:00:16.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.570+0000 7f11dd83b640 1 --2- 172.21.15.69:0/1556486241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11d8070270 0x7f11d813c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:16.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.570+0000 7f11dd03a640 1 --2- 172.21.15.69:0/1556486241 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11d81400a0 0x7f11d80ca310 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f11c800e960 tx=0x7f11c800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:16.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.570+0000 7f11ce7fc640 1 -- 172.21.15.69:0/1556486241 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c8017070 con 0x7f11d81400a0 2024-04-25T11:00:16.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.570+0000 7f11ce7fc640 1 -- 172.21.15.69:0/1556486241 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11c800c9a0 con 0x7f11d81400a0 2024-04-25T11:00:16.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.570+0000 7f11dfa9f640 1 -- 172.21.15.69:0/1556486241 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d80c7a90 con 0x7f11d81400a0 2024-04-25T11:00:16.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.570+0000 7f11ce7fc640 1 -- 172.21.15.69:0/1556486241 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c800cd70 con 0x7f11d81400a0 2024-04-25T11:00:16.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.570+0000 7f11dfa9f640 1 -- 172.21.15.69:0/1556486241 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d80cd1d0 con 0x7f11d81400a0 2024-04-25T11:00:16.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.571+0000 7f11dfa9f640 1 -- 172.21.15.69:0/1556486241 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11d8050740 con 0x7f11d81400a0 2024-04-25T11:00:16.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.571+0000 7f11ce7fc640 1 -- 172.21.15.69:0/1556486241 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f11c801a070 con 0x7f11d81400a0 2024-04-25T11:00:16.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.572+0000 7f11ce7fc640 1 --2- 172.21.15.69:0/1556486241 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f11bc0744e0 0x7f11bc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:16.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.572+0000 7f11ce7fc640 1 -- 172.21.15.69:0/1556486241 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f11c8094f60 con 0x7f11d81400a0 2024-04-25T11:00:16.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.572+0000 7f11dd83b640 1 --2- 172.21.15.69:0/1556486241 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f11bc0744e0 0x7f11bc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:16.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.572+0000 7f11dd83b640 1 --2- 172.21.15.69:0/1556486241 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f11bc0744e0 0x7f11bc0769f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f11d4005c00 tx=0x7f11d4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:16.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.577+0000 7f11ce7fc640 1 -- 172.21.15.69:0/1556486241 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f11c805f1c0 con 0x7f11d81400a0 2024-04-25T11:00:16.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.786+0000 7f11dfa9f640 1 -- 172.21.15.69:0/1556486241 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11d80d0d70 con 0x7f11d81400a0 2024-04-25T11:00:16.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.787+0000 7f11ce7fc640 1 -- 172.21.15.69:0/1556486241 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5177 (secure 0 0 0) 0x7f11c80634c0 con 0x7f11d81400a0 2024-04-25T11:00:16.787 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:00:16.787 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2223802951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":2223802951},{"type":"v1","addr":"172.21.15.146:6835","nonce":2223802951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4867,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/8243043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":8243043},{"type":"v1","addr":"172.21.15.146:6834","nonce":8243043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5031,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/3766456547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3766456547},{"type":"v1","addr":"172.21.15.69:6836","nonce":3766456547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:00:06.608766+0000","modified":"2024-04-25T11:00:09.621738+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5034},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5034":{"gid":5034,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6837/3153221129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":3153221129},{"type":"v1","addr":"172.21.15.69:6837","nonce":3153221129}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5034,"qdb_cluster":[5034]},"id":2}]} 2024-04-25T11:00:16.788 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 15 2024-04-25T11:00:16.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.788+0000 7f11dfa9f640 1 -- 172.21.15.69:0/1556486241 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f11bc0744e0 msgr2=0x7f11bc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:16.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.788+0000 7f11dfa9f640 1 --2- 172.21.15.69:0/1556486241 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f11bc0744e0 0x7f11bc0769f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f11d4005c00 tx=0x7f11d4009040 comp rx=0 tx=0).stop 2024-04-25T11:00:16.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.789+0000 7f11dfa9f640 1 -- 172.21.15.69:0/1556486241 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11d81400a0 msgr2=0x7f11d80ca310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:16.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.789+0000 7f11dfa9f640 1 --2- 172.21.15.69:0/1556486241 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11d81400a0 0x7f11d80ca310 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f11c800e960 tx=0x7f11c800ee20 comp rx=0 tx=0).stop 2024-04-25T11:00:16.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.789+0000 7f11dfa9f640 1 -- 172.21.15.69:0/1556486241 shutdown_connections 2024-04-25T11:00:16.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.789+0000 7f11dfa9f640 1 --2- 172.21.15.69:0/1556486241 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f11bc0744e0 0x7f11bc0769f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:16.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.789+0000 7f11dfa9f640 1 --2- 172.21.15.69:0/1556486241 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11d8142730 0x7f11d806f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:16.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.789+0000 7f11dfa9f640 1 --2- 172.21.15.69:0/1556486241 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11d81400a0 0x7f11d80ca310 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:16.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.789+0000 7f11dfa9f640 1 --2- 172.21.15.69:0/1556486241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11d8070270 0x7f11d813c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:16.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.789+0000 7f11dfa9f640 1 -- 172.21.15.69:0/1556486241 >> 172.21.15.69:0/1556486241 conn(0x7f11d8073050 msgr2=0x7f11d80b4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:16.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.789+0000 7f11dfa9f640 1 -- 172.21.15.69:0/1556486241 shutdown_connections 2024-04-25T11:00:16.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:16.789+0000 7f11dfa9f640 1 -- 172.21.15.69:0/1556486241 wait complete. 2024-04-25T11:00:17.203 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:00:06.608766+0000', 'modified': '2024-04-25T11:00:09.621738+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5034}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5034': {'gid': 5034, 'name': 'a', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.69:6837/3153221129', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6835', 'nonce': 3153221129}, {'type': 'v1', 'addr': '172.21.15.69:6837', 'nonce': 3153221129}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5034, 'qdb_cluster': [5034]} 2024-04-25T11:00:17.204 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:00:17.204 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T11:00:17.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.315+0000 7f0358214640 1 Processor -- start 2024-04-25T11:00:17.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.315+0000 7f0358214640 1 -- start start 2024-04-25T11:00:17.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.316+0000 7f0358214640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03500ce800 0x7f03500d1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:17.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.316+0000 7f0358214640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f03500d1620 0x7f0350162a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:17.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.316+0000 7f0358214640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03500b0770 con 0x7f0350162f60 2024-04-25T11:00:17.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.316+0000 7f0358214640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f03500b10a0 con 0x7f03500d1620 2024-04-25T11:00:17.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.316+0000 7f0358214640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f03500d2100 con 0x7f03500ce800 2024-04-25T11:00:17.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.317+0000 7f03557af640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f03500d1620 0x7f0350162a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:17.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.317+0000 7f0355fb0640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03500ce800 0x7f03500d1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:17.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.317+0000 7f03567b1640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f0350162f60 0x7f03500b1530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.69:53984/0 (socket says 172.21.15.69:53984) 2024-04-25T11:00:17.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.317+0000 7f03567b1640 1 -- 172.21.15.69:0/962807545 learned_addr learned my addr 172.21.15.69:0/962807545 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:17.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.317+0000 7f03557af640 1 --2- 172.21.15.69:0/962807545 >> v2:172.21.15.146:3300/0 conn(0x7f03500d1620 0x7f0350162a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.69:42940/0 (socket says 172.21.15.69:42940) 2024-04-25T11:00:17.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.317+0000 7f0355fb0640 1 -- 172.21.15.69:0/962807545 >> v2:172.21.15.146:3300/0 conn(0x7f03500d1620 msgr2=0x7f0350162a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:17.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.317+0000 7f0355fb0640 1 --2- 172.21.15.69:0/962807545 >> v2:172.21.15.146:3300/0 conn(0x7f03500d1620 0x7f0350162a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.317+0000 7f0355fb0640 1 -- 172.21.15.69:0/962807545 >> v1:172.21.15.146:6789/0 conn(0x7f0350162f60 legacy=0x7f03500b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:17.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.317+0000 7f0355fb0640 1 -- 172.21.15.69:0/962807545 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03500b0a50 con 0x7f03500ce800 2024-04-25T11:00:17.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.317+0000 7f03557af640 1 --2- 172.21.15.69:0/962807545 >> v2:172.21.15.146:3300/0 conn(0x7f03500d1620 0x7f0350162a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:00:17.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.318+0000 7f0355fb0640 1 --2- 172.21.15.69:0/962807545 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03500ce800 0x7f03500d1bc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f034c009790 tx=0x7f034c02a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24d1357732d60276 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:17.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.318+0000 7f0354fae640 1 -- 172.21.15.69:0/962807545 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c037070 con 0x7f03500ce800 2024-04-25T11:00:17.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.318+0000 7f0354fae640 1 -- 172.21.15.69:0/962807545 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f034c02adb0 con 0x7f03500ce800 2024-04-25T11:00:17.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.318+0000 7f0354fae640 1 -- 172.21.15.69:0/962807545 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c032660 con 0x7f03500ce800 2024-04-25T11:00:17.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.319+0000 7f0358214640 1 -- 172.21.15.69:0/962807545 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03500ce800 msgr2=0x7f03500d1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:17.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.319+0000 7f0358214640 1 --2- 172.21.15.69:0/962807545 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03500ce800 0x7f03500d1bc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f034c009790 tx=0x7f034c02a060 comp rx=0 tx=0).stop 2024-04-25T11:00:17.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.319+0000 7f0358214640 1 -- 172.21.15.69:0/962807545 shutdown_connections 2024-04-25T11:00:17.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.319+0000 7f0358214640 1 --2- 172.21.15.69:0/962807545 >> v2:172.21.15.146:3300/0 conn(0x7f03500d1620 0x7f0350162a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.319+0000 7f0358214640 1 --2- 172.21.15.69:0/962807545 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03500ce800 0x7f03500d1bc0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.319+0000 7f0358214640 1 -- 172.21.15.69:0/962807545 >> 172.21.15.69:0/962807545 conn(0x7f03500d3a50 msgr2=0x7f03500d3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:17.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.319+0000 7f0358214640 1 -- 172.21.15.69:0/962807545 shutdown_connections 2024-04-25T11:00:17.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.319+0000 7f0358214640 1 -- 172.21.15.69:0/962807545 wait complete. 2024-04-25T11:00:17.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.320+0000 7f0358214640 1 Processor -- start 2024-04-25T11:00:17.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.320+0000 7f0358214640 1 -- start start 2024-04-25T11:00:17.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.322+0000 7f0358214640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03500ce800 0x7f03500cd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:17.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.322+0000 7f0358214640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03500d1620 0x7f0350159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:17.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.322+0000 7f0358214640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0350162f60 0x7f03501595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:17.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.322+0000 7f0358214640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0350163370 con 0x7f03500d1620 2024-04-25T11:00:17.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.322+0000 7f0358214640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f03500b51c0 con 0x7f0350162f60 2024-04-25T11:00:17.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.322+0000 7f0358214640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f03501634f0 con 0x7f03500ce800 2024-04-25T11:00:17.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.322+0000 7f03557af640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03500d1620 0x7f0350159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:17.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.322+0000 7f03557af640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03500d1620 0x7f0350159070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51046/0 (socket says 172.21.15.69:51046) 2024-04-25T11:00:17.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.322+0000 7f03557af640 1 -- 172.21.15.69:0/3068034221 learned_addr learned my addr 172.21.15.69:0/3068034221 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:17.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.322+0000 7f0355fb0640 1 --2- 172.21.15.69:0/3068034221 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03500ce800 0x7f03500cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:17.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.322+0000 7f03567b1640 1 --2- 172.21.15.69:0/3068034221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0350162f60 0x7f03501595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:17.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.322+0000 7f03557af640 1 -- 172.21.15.69:0/3068034221 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03500ce800 msgr2=0x7f03500cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:17.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.322+0000 7f03557af640 1 --2- 172.21.15.69:0/3068034221 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03500ce800 0x7f03500cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.322+0000 7f03557af640 1 -- 172.21.15.69:0/3068034221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0350162f60 msgr2=0x7f03501595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:17.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.322+0000 7f03557af640 1 --2- 172.21.15.69:0/3068034221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0350162f60 0x7f03501595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.322+0000 7f03557af640 1 -- 172.21.15.69:0/3068034221 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03500b46a0 con 0x7f03500d1620 2024-04-25T11:00:17.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.322+0000 7f03557af640 1 --2- 172.21.15.69:0/3068034221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03500d1620 0x7f0350159070 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f034000e940 tx=0x7f034000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:17.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.322+0000 7f03467fc640 1 -- 172.21.15.69:0/3068034221 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340017070 con 0x7f03500d1620 2024-04-25T11:00:17.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.323+0000 7f03467fc640 1 -- 172.21.15.69:0/3068034221 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f034000ca00 con 0x7f03500d1620 2024-04-25T11:00:17.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.323+0000 7f03467fc640 1 -- 172.21.15.69:0/3068034221 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034000cdd0 con 0x7f03500d1620 2024-04-25T11:00:17.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.323+0000 7f0358214640 1 -- 172.21.15.69:0/3068034221 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03500b15d0 con 0x7f03500d1620 2024-04-25T11:00:17.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.323+0000 7f0358214640 1 -- 172.21.15.69:0/3068034221 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03500b1b90 con 0x7f03500d1620 2024-04-25T11:00:17.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.324+0000 7f03467fc640 1 -- 172.21.15.69:0/3068034221 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f034001a070 con 0x7f03500d1620 2024-04-25T11:00:17.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.325+0000 7f03467fc640 1 --2- 172.21.15.69:0/3068034221 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f03340744b0 0x7f03340769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:17.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.325+0000 7f0355fb0640 1 --2- 172.21.15.69:0/3068034221 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f03340744b0 0x7f03340769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:17.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.325+0000 7f03467fc640 1 -- 172.21.15.69:0/3068034221 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f0340095400 con 0x7f03500d1620 2024-04-25T11:00:17.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.325+0000 7f0355fb0640 1 --2- 172.21.15.69:0/3068034221 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f03340744b0 0x7f03340769c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f034c034000 tx=0x7f034c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:17.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.326+0000 7f035f19c740 1 Processor -- start 2024-04-25T11:00:17.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.326+0000 7f035f19c740 1 -- start start 2024-04-25T11:00:17.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.327+0000 7f035f19c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfb8070270 0x55bfb840c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:17.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.327+0000 7f035f19c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfb80473b0 0x55bfb84394a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:17.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.327+0000 7f035f19c740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfb804f590 0x55bfb8412260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:17.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.327+0000 7f035f19c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bfb8376740 con 0x55bfb8070270 2024-04-25T11:00:17.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.327+0000 7f035f19c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55bfb83123a0 con 0x55bfb80473b0 2024-04-25T11:00:17.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.327+0000 7f035f19c740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x55bfb8372700 con 0x55bfb804f590 2024-04-25T11:00:17.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.327+0000 7f0355fb0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfb8070270 0x55bfb840c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:17.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.327+0000 7f0355fb0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfb8070270 0x55bfb840c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51060/0 (socket says 172.21.15.69:51060) 2024-04-25T11:00:17.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.327+0000 7f0355fb0640 1 -- 172.21.15.69:0/4062759198 learned_addr learned my addr 172.21.15.69:0/4062759198 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:17.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.327+0000 7f03567b1640 1 --2- 172.21.15.69:0/4062759198 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfb804f590 0x55bfb8412260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:17.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.327+0000 7f03557af640 1 --2- 172.21.15.69:0/4062759198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfb80473b0 0x55bfb84394a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:17.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.328+0000 7f0355fb0640 1 -- 172.21.15.69:0/4062759198 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfb804f590 msgr2=0x55bfb8412260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:17.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.328+0000 7f0355fb0640 1 --2- 172.21.15.69:0/4062759198 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfb804f590 0x55bfb8412260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.328+0000 7f0355fb0640 1 -- 172.21.15.69:0/4062759198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfb80473b0 msgr2=0x55bfb84394a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:17.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.328+0000 7f0355fb0640 1 --2- 172.21.15.69:0/4062759198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfb80473b0 0x55bfb84394a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.328+0000 7f0355fb0640 1 -- 172.21.15.69:0/4062759198 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfb832e750 con 0x55bfb8070270 2024-04-25T11:00:17.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.328+0000 7f0355fb0640 1 --2- 172.21.15.69:0/4062759198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfb8070270 0x55bfb840c690 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f034c009880 tx=0x7f034c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57c8912b0271fd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:17.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.328+0000 7f03337fe640 1 -- 172.21.15.69:0/4062759198 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c037040 con 0x55bfb8070270 2024-04-25T11:00:17.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.328+0000 7f03337fe640 1 -- 172.21.15.69:0/4062759198 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f034c04e070 con 0x55bfb8070270 2024-04-25T11:00:17.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.328+0000 7f03337fe640 1 -- 172.21.15.69:0/4062759198 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c03d670 con 0x55bfb8070270 2024-04-25T11:00:17.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.329+0000 7f035f19c740 1 -- 172.21.15.69:0/4062759198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfb8070270 msgr2=0x55bfb840c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:17.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.329+0000 7f035f19c740 1 --2- 172.21.15.69:0/4062759198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfb8070270 0x55bfb840c690 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f034c009880 tx=0x7f034c02c000 comp rx=0 tx=0).stop 2024-04-25T11:00:17.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.329+0000 7f035f19c740 1 -- 172.21.15.69:0/4062759198 shutdown_connections 2024-04-25T11:00:17.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.329+0000 7f035f19c740 1 --2- 172.21.15.69:0/4062759198 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfb804f590 0x55bfb8412260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.329+0000 7f035f19c740 1 --2- 172.21.15.69:0/4062759198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfb80473b0 0x55bfb84394a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.329+0000 7f035f19c740 1 --2- 172.21.15.69:0/4062759198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfb8070270 0x55bfb840c690 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.329+0000 7f035f19c740 1 -- 172.21.15.69:0/4062759198 >> 172.21.15.69:0/4062759198 conn(0x55bfb80643c0 msgr2=0x55bfb84119f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:17.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.329+0000 7f035f19c740 1 -- 172.21.15.69:0/4062759198 shutdown_connections 2024-04-25T11:00:17.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.329+0000 7f035f19c740 1 -- 172.21.15.69:0/4062759198 wait complete. 2024-04-25T11:00:17.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.330+0000 7f035f19c740 1 -- start start 2024-04-25T11:00:17.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.330+0000 7f035f19c740 1 Processor -- start 2024-04-25T11:00:17.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.331+0000 7f035f19c740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfb80473b0 0x55bfb8456060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:17.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.331+0000 7f035f19c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfb804f590 0x55bfb8412fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:17.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.331+0000 7f035f19c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfb8070270 0x55bfb8413510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:17.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.331+0000 7f035f19c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bfb82fdfe0 con 0x55bfb8070270 2024-04-25T11:00:17.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.331+0000 7f035f19c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55bfb843cd40 con 0x55bfb804f590 2024-04-25T11:00:17.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.331+0000 7f035f19c740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x55bfb84549d0 con 0x55bfb80473b0 2024-04-25T11:00:17.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.331+0000 7f035f19c740 10 client.0 _ll_register_callbacks cb 0x55bfb8314150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:00:17.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.331+0000 7f03567b1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfb8070270 0x55bfb8413510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:17.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.331+0000 7f03567b1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfb8070270 0x55bfb8413510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51066/0 (socket says 172.21.15.69:51066) 2024-04-25T11:00:17.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.331+0000 7f03567b1640 1 -- 172.21.15.69:0/3684952317 learned_addr learned my addr 172.21.15.69:0/3684952317 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:17.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.331+0000 7f0355fb0640 1 --2- 172.21.15.69:0/3684952317 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfb80473b0 0x55bfb8456060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:17.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.331+0000 7f03557af640 1 --2- 172.21.15.69:0/3684952317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfb804f590 0x55bfb8412fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:17.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.332+0000 7f03567b1640 1 -- 172.21.15.69:0/3684952317 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfb80473b0 msgr2=0x55bfb8456060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:17.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.332+0000 7f03567b1640 1 --2- 172.21.15.69:0/3684952317 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfb80473b0 0x55bfb8456060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.332+0000 7f03567b1640 1 -- 172.21.15.69:0/3684952317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfb804f590 msgr2=0x55bfb8412fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:17.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.332+0000 7f03567b1640 1 --2- 172.21.15.69:0/3684952317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfb804f590 0x55bfb8412fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.332+0000 7f03567b1640 1 -- 172.21.15.69:0/3684952317 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfb80882d0 con 0x55bfb8070270 2024-04-25T11:00:17.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.332+0000 7f03567b1640 1 --2- 172.21.15.69:0/3684952317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfb8070270 0x55bfb8413510 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0348007770 tx=0x7f0348007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53297cc44515443 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:17.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.332+0000 7f0330ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-25T11:00:17.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.332+0000 7f0330ff9640 1 -- 172.21.15.69:0/3684952317 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0348019030 con 0x55bfb8070270 2024-04-25T11:00:17.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.332+0000 7f0330ff9640 1 -- 172.21.15.69:0/3684952317 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f0348010040 con 0x55bfb8070270 2024-04-25T11:00:17.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.332+0000 7f035f19c740 1 -- 172.21.15.69:0/3684952317 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f034c04e070 con 0x55bfb8070270 2024-04-25T11:00:17.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.332+0000 7f0330ff9640 1 -- 172.21.15.69:0/3684952317 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7f0348022050 con 0x55bfb8070270 2024-04-25T11:00:17.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.332+0000 7f035f19c740 10 client.5208 fetch_fsmap learned FSMap version 15 2024-04-25T11:00:17.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.332+0000 7f035f19c740 1 -- 172.21.15.69:0/3684952317 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) -- 0x55bfb81526b0 con 0x55bfb8070270 2024-04-25T11:00:17.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.332+0000 7f0330ff9640 1 -- 172.21.15.69:0/3684952317 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f034c04e070 con 0x55bfb8070270 2024-04-25T11:00:17.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.332+0000 7f0330ff9640 1 -- 172.21.15.69:0/3684952317 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 15) ==== 3494+0+0 (secure 0 0 0) 0x7f0348025020 con 0x55bfb8070270 2024-04-25T11:00:17.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.332+0000 7f035f19c740 10 client.5208 fetch_fsmap finished waiting for FSMap version 15 2024-04-25T11:00:17.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.332+0000 7f035f19c740 10 client.5208 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:00:17.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.332+0000 7f035f19c740 20 client.5208 populate_metadata read hostname 'smithi069' 2024-04-25T11:00:17.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.332+0000 7f035f19c740 1 --2- 172.21.15.69:0/3684952317 >> [v2:172.21.15.69:6835/3153221129,v1:172.21.15.69:6837/3153221129] conn(0x55bfb802ee40 0x55bfb843d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:17.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.332+0000 7f035f19c740 4 client.5208 mds_command: new command op to 5034 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:00:17.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.332+0000 7f035f19c740 1 -- 172.21.15.69:0/3684952317 --> [v2:172.21.15.69:6835/3153221129,v1:172.21.15.69:6837/3153221129] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfb8384350 con 0x55bfb802ee40 2024-04-25T11:00:17.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.332+0000 7f0355fb0640 1 --2- 172.21.15.69:0/3684952317 >> [v2:172.21.15.69:6835/3153221129,v1:172.21.15.69:6837/3153221129] conn(0x55bfb802ee40 0x55bfb843d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:17.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.333+0000 7f0330ff9640 10 client.5208 ms_handle_connect on v2:172.21.15.69:6835/3153221129 2024-04-25T11:00:17.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.333+0000 7f0355fb0640 1 --2- 172.21.15.69:0/3684952317 >> [v2:172.21.15.69:6835/3153221129,v1:172.21.15.69:6837/3153221129] conn(0x55bfb802ee40 0x55bfb843d270 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f84fa5f808f7b287 server_cookie=e5e86a07e3743aa1 in_seq=0 out_seq=0 2024-04-25T11:00:17.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.334+0000 7f0330ff9640 1 -- 172.21.15.69:0/3684952317 <== mds.0 v2:172.21.15.69:6835/3153221129 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bfb8384350 con 0x55bfb802ee40 2024-04-25T11:00:17.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.334+0000 7f0330ff9640 10 client.5208 handle_command_reply: tid=0 2024-04-25T11:00:17.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.334+0000 7f035f19c740 1 client.5208 shutdown 2024-04-25T11:00:17.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.334+0000 7f035f19c740 1 -- 172.21.15.69:0/3684952317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfb8070270 msgr2=0x55bfb8413510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:17.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.334+0000 7f035f19c740 1 --2- 172.21.15.69:0/3684952317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfb8070270 0x55bfb8413510 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0348007770 tx=0x7f0348007860 comp rx=0 tx=0).stop 2024-04-25T11:00:17.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.334+0000 7f035f19c740 1 -- 172.21.15.69:0/3684952317 shutdown_connections 2024-04-25T11:00:17.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.334+0000 7f035f19c740 1 --2- 172.21.15.69:0/3684952317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfb8070270 0x55bfb8413510 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.334+0000 7f035f19c740 1 --2- 172.21.15.69:0/3684952317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfb804f590 0x55bfb8412fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.334+0000 7f035f19c740 1 --2- 172.21.15.69:0/3684952317 >> [v2:172.21.15.69:6835/3153221129,v1:172.21.15.69:6837/3153221129] conn(0x55bfb802ee40 0x55bfb843d270 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.334+0000 7f035f19c740 1 --2- 172.21.15.69:0/3684952317 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfb80473b0 0x55bfb8456060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.334+0000 7f0330ff9640 0 client.5208 ms_handle_reset on v2:172.21.15.69:6835/3153221129 2024-04-25T11:00:17.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.334+0000 7f035f19c740 1 -- 172.21.15.69:0/3684952317 >> 172.21.15.69:0/3684952317 conn(0x55bfb80643c0 msgr2=0x55bfb8410c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:17.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.334+0000 7f035f19c740 1 -- 172.21.15.69:0/3684952317 shutdown_connections 2024-04-25T11:00:17.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.334+0000 7f035f19c740 1 -- 172.21.15.69:0/3684952317 wait complete. 2024-04-25T11:00:17.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.334+0000 7f035f19c740 20 client.5208 trim_cache size 0 max 16384 2024-04-25T11:00:17.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.352+0000 7f035f19c740 1 Processor -- start 2024-04-25T11:00:17.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.352+0000 7f035f19c740 1 -- start start 2024-04-25T11:00:17.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.354+0000 7f035f19c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfb80643c0 0x55bfb847e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:17.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.354+0000 7f035f19c740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfb80473b0 0x55bfb847eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:17.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.354+0000 7f035f19c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfb804f590 0x55bfb843d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:17.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.354+0000 7f035f19c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f035006f580 con 0x55bfb80643c0 2024-04-25T11:00:17.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.354+0000 7f035f19c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f035006e8e0 con 0x55bfb804f590 2024-04-25T11:00:17.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.354+0000 7f035f19c740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f035006ef30 con 0x55bfb80473b0 2024-04-25T11:00:17.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.354+0000 7f0355fb0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfb80643c0 0x55bfb847e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:17.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.354+0000 7f0355fb0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfb80643c0 0x55bfb847e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51080/0 (socket says 172.21.15.69:51080) 2024-04-25T11:00:17.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.354+0000 7f0355fb0640 1 -- 172.21.15.69:0/3849564390 learned_addr learned my addr 172.21.15.69:0/3849564390 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:17.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.354+0000 7f0355fb0640 1 -- 172.21.15.69:0/3849564390 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfb80473b0 msgr2=0x55bfb847eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:17.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.354+0000 7f0355fb0640 1 --2- 172.21.15.69:0/3849564390 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfb80473b0 0x55bfb847eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.354+0000 7f0355fb0640 1 -- 172.21.15.69:0/3849564390 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfb804f590 msgr2=0x55bfb843d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:17.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.354+0000 7f0355fb0640 1 --2- 172.21.15.69:0/3849564390 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfb804f590 0x55bfb843d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.354+0000 7f0355fb0640 1 -- 172.21.15.69:0/3849564390 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfb8438e00 con 0x55bfb80643c0 2024-04-25T11:00:17.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.354+0000 7f0355fb0640 1 --2- 172.21.15.69:0/3849564390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfb80643c0 0x55bfb847e660 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f034c035040 tx=0x7f034c009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e3457c404447c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:17.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.354+0000 7f0332ffd640 1 -- 172.21.15.69:0/3849564390 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c03d9f0 con 0x55bfb80643c0 2024-04-25T11:00:17.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.354+0000 7f0332ffd640 1 -- 172.21.15.69:0/3849564390 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f034c04e040 con 0x55bfb80643c0 2024-04-25T11:00:17.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.354+0000 7f0332ffd640 1 -- 172.21.15.69:0/3849564390 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c03dd90 con 0x55bfb80643c0 2024-04-25T11:00:17.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.355+0000 7f035f19c740 1 -- 172.21.15.69:0/3849564390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfb80643c0 msgr2=0x55bfb847e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:17.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.355+0000 7f035f19c740 1 --2- 172.21.15.69:0/3849564390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfb80643c0 0x55bfb847e660 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f034c035040 tx=0x7f034c009110 comp rx=0 tx=0).stop 2024-04-25T11:00:17.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.355+0000 7f035f19c740 1 -- 172.21.15.69:0/3849564390 shutdown_connections 2024-04-25T11:00:17.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.355+0000 7f035f19c740 1 --2- 172.21.15.69:0/3849564390 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfb804f590 0x55bfb843d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.355+0000 7f035f19c740 1 --2- 172.21.15.69:0/3849564390 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfb80473b0 0x55bfb847eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.355+0000 7f035f19c740 1 --2- 172.21.15.69:0/3849564390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfb80643c0 0x55bfb847e660 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.355+0000 7f035f19c740 1 -- 172.21.15.69:0/3849564390 >> 172.21.15.69:0/3849564390 conn(0x55bfb840e270 msgr2=0x55bfb840b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:17.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.355+0000 7f035f19c740 1 -- 172.21.15.69:0/3849564390 shutdown_connections 2024-04-25T11:00:17.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.355+0000 7f035f19c740 1 -- 172.21.15.69:0/3849564390 wait complete. 2024-04-25T11:00:17.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.356+0000 7f035f19c740 1 -- start start 2024-04-25T11:00:17.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.356+0000 7f035f19c740 1 Processor -- start 2024-04-25T11:00:17.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.357+0000 7f035f19c740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfb80473b0 0x55bfb84beb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:17.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.357+0000 7f035f19c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfb804f590 0x55bfb843d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:17.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.357+0000 7f035f19c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfb80643c0 0x55bfb8455b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:17.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.357+0000 7f035f19c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bfb8434950 con 0x55bfb80643c0 2024-04-25T11:00:17.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.357+0000 7f035f19c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55bfb8434e30 con 0x55bfb804f590 2024-04-25T11:00:17.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.357+0000 7f035f19c740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x55bfb8456830 con 0x55bfb80473b0 2024-04-25T11:00:17.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.357+0000 7f035f19c740 10 client.0 _ll_register_callbacks cb 0x7f0348001d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:00:17.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.358+0000 7f03567b1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfb80643c0 0x55bfb8455b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:17.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.358+0000 7f03567b1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfb80643c0 0x55bfb8455b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51088/0 (socket says 172.21.15.69:51088) 2024-04-25T11:00:17.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.358+0000 7f03567b1640 1 -- 172.21.15.69:0/1807796975 learned_addr learned my addr 172.21.15.69:0/1807796975 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:17.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.358+0000 7f0355fb0640 1 --2- 172.21.15.69:0/1807796975 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfb80473b0 0x55bfb84beb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:17.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.358+0000 7f03557af640 1 --2- 172.21.15.69:0/1807796975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfb804f590 0x55bfb843d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:17.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.358+0000 7f03567b1640 1 -- 172.21.15.69:0/1807796975 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfb80473b0 msgr2=0x55bfb84beb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:17.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.358+0000 7f03567b1640 1 --2- 172.21.15.69:0/1807796975 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfb80473b0 0x55bfb84beb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.358+0000 7f03567b1640 1 -- 172.21.15.69:0/1807796975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfb804f590 msgr2=0x55bfb843d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:17.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.358+0000 7f03567b1640 1 --2- 172.21.15.69:0/1807796975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfb804f590 0x55bfb843d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.358+0000 7f03567b1640 1 -- 172.21.15.69:0/1807796975 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfb8410ed0 con 0x55bfb80643c0 2024-04-25T11:00:17.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.358+0000 7f0355fb0640 1 --2- 172.21.15.69:0/1807796975 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfb80473b0 0x55bfb84beb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:17.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.358+0000 7f03557af640 1 --2- 172.21.15.69:0/1807796975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfb804f590 0x55bfb843d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:17.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.358+0000 7f03567b1640 1 --2- 172.21.15.69:0/1807796975 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfb80643c0 0x55bfb8455b80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f03480070e0 tx=0x7f0348001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104e2c60374e4620 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:17.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.358+0000 7f0330ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-25T11:00:17.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.358+0000 7f0330ff9640 1 -- 172.21.15.69:0/1807796975 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034802a020 con 0x55bfb80643c0 2024-04-25T11:00:17.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.358+0000 7f0330ff9640 1 -- 172.21.15.69:0/1807796975 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f0348015880 con 0x55bfb80643c0 2024-04-25T11:00:17.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.358+0000 7f035f19c740 1 -- 172.21.15.69:0/1807796975 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f034c04e040 con 0x55bfb80643c0 2024-04-25T11:00:17.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.358+0000 7f0330ff9640 1 -- 172.21.15.69:0/1807796975 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7f0348031050 con 0x55bfb80643c0 2024-04-25T11:00:17.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.358+0000 7f035f19c740 10 client.5214 fetch_fsmap learned FSMap version 15 2024-04-25T11:00:17.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.358+0000 7f035f19c740 1 -- 172.21.15.69:0/1807796975 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) -- 0x55bfb8412e30 con 0x55bfb80643c0 2024-04-25T11:00:17.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.359+0000 7f0330ff9640 1 -- 172.21.15.69:0/1807796975 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f034c04e040 con 0x55bfb80643c0 2024-04-25T11:00:17.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.359+0000 7f0330ff9640 1 -- 172.21.15.69:0/1807796975 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 15) ==== 3494+0+0 (secure 0 0 0) 0x7f0348034020 con 0x55bfb80643c0 2024-04-25T11:00:17.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.359+0000 7f035f19c740 10 client.5214 fetch_fsmap finished waiting for FSMap version 15 2024-04-25T11:00:17.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.359+0000 7f035f19c740 10 client.5214 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:00:17.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.359+0000 7f035f19c740 20 client.5214 populate_metadata read hostname 'smithi069' 2024-04-25T11:00:17.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.359+0000 7f035f19c740 1 --2- 172.21.15.69:0/1807796975 >> [v2:172.21.15.69:6835/3153221129,v1:172.21.15.69:6837/3153221129] conn(0x55bfb8004280 0x55bfb8473fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:17.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.359+0000 7f035f19c740 4 client.5214 mds_command: new command op to 5034 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:00:17.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.359+0000 7f035f19c740 1 -- 172.21.15.69:0/1807796975 --> [v2:172.21.15.69:6835/3153221129,v1:172.21.15.69:6837/3153221129] -- command(tid 0: {"prefix": "status"}) -- 0x55bfb837e1c0 con 0x55bfb8004280 2024-04-25T11:00:17.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.359+0000 7f0355fb0640 1 --2- 172.21.15.69:0/1807796975 >> [v2:172.21.15.69:6835/3153221129,v1:172.21.15.69:6837/3153221129] conn(0x55bfb8004280 0x55bfb8473fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:17.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.359+0000 7f0330ff9640 10 client.5214 ms_handle_connect on v2:172.21.15.69:6835/3153221129 2024-04-25T11:00:17.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.359+0000 7f0355fb0640 1 --2- 172.21.15.69:0/1807796975 >> [v2:172.21.15.69:6835/3153221129,v1:172.21.15.69:6837/3153221129] conn(0x55bfb8004280 0x55bfb8473fc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=225c65c4b6c99ed8 server_cookie=e645d946ce63a037 in_seq=0 out_seq=0 2024-04-25T11:00:17.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.360+0000 7f0330ff9640 1 -- 172.21.15.69:0/1807796975 <== mds.0 v2:172.21.15.69:6835/3153221129 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55bfb837e1c0 con 0x55bfb8004280 2024-04-25T11:00:17.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.360+0000 7f0330ff9640 10 client.5214 handle_command_reply: tid=0 2024-04-25T11:00:17.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.360+0000 7f035f19c740 1 client.5214 shutdown 2024-04-25T11:00:17.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.360+0000 7f035f19c740 1 -- 172.21.15.69:0/1807796975 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfb80643c0 msgr2=0x55bfb8455b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:17.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.360+0000 7f035f19c740 1 --2- 172.21.15.69:0/1807796975 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfb80643c0 0x55bfb8455b80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f03480070e0 tx=0x7f0348001250 comp rx=0 tx=0).stop 2024-04-25T11:00:17.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.360+0000 7f035f19c740 1 -- 172.21.15.69:0/1807796975 shutdown_connections 2024-04-25T11:00:17.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.360+0000 7f035f19c740 1 --2- 172.21.15.69:0/1807796975 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfb80643c0 0x55bfb8455b80 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.360+0000 7f035f19c740 1 --2- 172.21.15.69:0/1807796975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfb804f590 0x55bfb843d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.360+0000 7f035f19c740 1 --2- 172.21.15.69:0/1807796975 >> [v2:172.21.15.69:6835/3153221129,v1:172.21.15.69:6837/3153221129] conn(0x55bfb8004280 0x55bfb8473fc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.360+0000 7f035f19c740 1 --2- 172.21.15.69:0/1807796975 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bfb80473b0 0x55bfb84beb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.360+0000 7f0330ff9640 0 client.5214 ms_handle_reset on v2:172.21.15.69:6835/3153221129 2024-04-25T11:00:17.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.360+0000 7f035f19c740 1 -- 172.21.15.69:0/1807796975 >> 172.21.15.69:0/1807796975 conn(0x55bfb840e270 msgr2=0x55bfb8457f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:17.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.360+0000 7f035f19c740 1 -- 172.21.15.69:0/1807796975 shutdown_connections 2024-04-25T11:00:17.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.360+0000 7f035f19c740 1 -- 172.21.15.69:0/1807796975 wait complete. 2024-04-25T11:00:17.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.360+0000 7f035f19c740 20 client.5214 trim_cache size 0 max 16384 2024-04-25T11:00:17.367 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T11:00:17.367 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "ed0c8fe4-1bbb-4705-be72-eeea5bef859e", 2024-04-25T11:00:17.367 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 0, 2024-04-25T11:00:17.367 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 5034, 2024-04-25T11:00:17.367 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-04-25T11:00:17.367 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-04-25T11:00:17.367 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-04-25T11:00:17.367 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 10.108625601, 2024-04-25T11:00:17.367 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 15, 2024-04-25T11:00:17.367 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 31, 2024-04-25T11:00:17.368 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 31, 2024-04-25T11:00:17.368 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 17.144064018000002 2024-04-25T11:00:17.368 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T11:00:17.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.360+0000 7f0358214640 1 -- 172.21.15.69:0/3068034221 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f03340744b0 msgr2=0x7f03340769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:17.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.360+0000 7f0358214640 1 --2- 172.21.15.69:0/3068034221 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f03340744b0 0x7f03340769c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f034c034000 tx=0x7f034c02e000 comp rx=0 tx=0).stop 2024-04-25T11:00:17.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.360+0000 7f0358214640 1 -- 172.21.15.69:0/3068034221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03500d1620 msgr2=0x7f0350159070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:17.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.360+0000 7f0358214640 1 --2- 172.21.15.69:0/3068034221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03500d1620 0x7f0350159070 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f034000e940 tx=0x7f034000ee00 comp rx=0 tx=0).stop 2024-04-25T11:00:17.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.360+0000 7f0358214640 1 -- 172.21.15.69:0/3068034221 shutdown_connections 2024-04-25T11:00:17.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.360+0000 7f0358214640 1 --2- 172.21.15.69:0/3068034221 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f03340744b0 0x7f03340769c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.360+0000 7f0358214640 1 --2- 172.21.15.69:0/3068034221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0350162f60 0x7f03501595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.360+0000 7f0358214640 1 --2- 172.21.15.69:0/3068034221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03500d1620 0x7f0350159070 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.360+0000 7f0358214640 1 --2- 172.21.15.69:0/3068034221 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f03500ce800 0x7f03500cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.360+0000 7f0358214640 1 -- 172.21.15.69:0/3068034221 >> 172.21.15.69:0/3068034221 conn(0x7f03500d3a50 msgr2=0x7f0350156b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:17.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.361+0000 7f0358214640 1 -- 172.21.15.69:0/3068034221 shutdown_connections 2024-04-25T11:00:17.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.361+0000 7f0358214640 1 -- 172.21.15.69:0/3068034221 wait complete. 2024-04-25T11:00:17.372 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:00:17.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.478+0000 7f19970b1640 1 Processor -- start 2024-04-25T11:00:17.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.478+0000 7f19970b1640 1 -- start start 2024-04-25T11:00:17.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.480+0000 7f19970b1640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f199006def0 0x7f19900cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:17.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.480+0000 7f19970b1640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19900b3af0 con 0x7f19900d1850 2024-04-25T11:00:17.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.480+0000 7f19970b1640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1990164320 con 0x7f19900b7750 2024-04-25T11:00:17.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.480+0000 7f19970b1640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f19900b2f70 con 0x7f199006def0 2024-04-25T11:00:17.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.480+0000 7f198ffff640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f19900d1850 0x7f199006d7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:35540/0 (socket says 172.21.15.69:35540) 2024-04-25T11:00:17.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.480+0000 7f198ffff640 1 -- 172.21.15.69:0/3919886321 learned_addr learned my addr 172.21.15.69:0/3919886321 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:17.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.480+0000 7f199564e640 1 --2- 172.21.15.69:0/3919886321 >> v2:172.21.15.146:3300/0 conn(0x7f199006def0 0x7f19900cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:17.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.480+0000 7f198f7fe640 1 -- 172.21.15.69:0/3919886321 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2494551081 0 0) 0x7f19900b3af0 con 0x7f19900d1850 2024-04-25T11:00:17.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.480+0000 7f198f7fe640 1 -- 172.21.15.69:0/3919886321 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1978003520 con 0x7f19900d1850 2024-04-25T11:00:17.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.480+0000 7f198f7fe640 1 -- 172.21.15.69:0/3919886321 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3335632423 0 0) 0x7f1978003520 con 0x7f19900d1850 2024-04-25T11:00:17.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.480+0000 7f198f7fe640 1 -- 172.21.15.69:0/3919886321 >> v2:172.21.15.146:3300/0 conn(0x7f199006def0 msgr2=0x7f19900cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:17.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.480+0000 7f198f7fe640 1 --2- 172.21.15.69:0/3919886321 >> v2:172.21.15.146:3300/0 conn(0x7f199006def0 0x7f19900cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.480+0000 7f198f7fe640 1 -- 172.21.15.69:0/3919886321 >> v1:172.21.15.146:6789/0 conn(0x7f19900b7750 legacy=0x7f1990164800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:17.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.480+0000 7f198f7fe640 1 -- 172.21.15.69:0/3919886321 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19900b3250 con 0x7f19900d1850 2024-04-25T11:00:17.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.480+0000 7f198f7fe640 1 -- 172.21.15.69:0/3919886321 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7f1984001670 con 0x7f19900d1850 2024-04-25T11:00:17.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.480+0000 7f198f7fe640 1 -- 172.21.15.69:0/3919886321 >> v1:172.21.15.69:6789/0 conn(0x7f19900d1850 legacy=0x7f199006d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:17.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.480+0000 7f198f7fe640 1 --2- 172.21.15.69:0/3919886321 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19780039b0 0x7f1978003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:17.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.480+0000 7f198f7fe640 1 --2- 172.21.15.69:0/3919886321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1978004800 0x7f1978004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:17.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.480+0000 7f198f7fe640 1 --2- 172.21.15.69:0/3919886321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1978005250 0x7f197800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:17.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.480+0000 7f198f7fe640 1 -- 172.21.15.69:0/3919886321 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f19900b2f70 con 0x7f1978005250 2024-04-25T11:00:17.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.480+0000 7f198f7fe640 1 -- 172.21.15.69:0/3919886321 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f197800c1e0 con 0x7f1978004800 2024-04-25T11:00:17.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.480+0000 7f198f7fe640 1 -- 172.21.15.69:0/3919886321 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f197800c3e0 con 0x7f19780039b0 2024-04-25T11:00:17.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.481+0000 7f199564e640 1 --2- 172.21.15.69:0/3919886321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1978005250 0x7f197800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:17.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.481+0000 7f198ffff640 1 --2- 172.21.15.69:0/3919886321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1978004800 0x7f1978004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:17.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.481+0000 7f1994e4d640 1 --2- 172.21.15.69:0/3919886321 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19780039b0 0x7f1978003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:17.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.481+0000 7f199564e640 1 -- 172.21.15.69:0/3919886321 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19780039b0 msgr2=0x7f1978003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:17.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.481+0000 7f199564e640 1 --2- 172.21.15.69:0/3919886321 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19780039b0 0x7f1978003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.481+0000 7f199564e640 1 -- 172.21.15.69:0/3919886321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1978004800 msgr2=0x7f1978004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:17.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.481+0000 7f199564e640 1 --2- 172.21.15.69:0/3919886321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1978004800 0x7f1978004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.481+0000 7f199564e640 1 -- 172.21.15.69:0/3919886321 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1978003520 con 0x7f1978005250 2024-04-25T11:00:17.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.481+0000 7f1994e4d640 1 --2- 172.21.15.69:0/3919886321 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19780039b0 0x7f1978003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:17.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.481+0000 7f199564e640 1 -- 172.21.15.69:0/3919886321 reap_dead start 2024-04-25T11:00:17.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.481+0000 7f199564e640 1 --2- 172.21.15.69:0/3919886321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1978005250 0x7f197800b9c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1988009fc0 tx=0x7f1978005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed41f6b15651e950 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:17.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.482+0000 7f198f7fe640 1 -- 172.21.15.69:0/3919886321 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1988035020 con 0x7f1978005250 2024-04-25T11:00:17.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.482+0000 7f198f7fe640 1 -- 172.21.15.69:0/3919886321 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1988003bd0 con 0x7f1978005250 2024-04-25T11:00:17.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.482+0000 7f198f7fe640 1 -- 172.21.15.69:0/3919886321 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1988009c10 con 0x7f1978005250 2024-04-25T11:00:17.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.482+0000 7f19970b1640 1 -- 172.21.15.69:0/3919886321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1978005250 msgr2=0x7f197800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:17.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.482+0000 7f19970b1640 1 --2- 172.21.15.69:0/3919886321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1978005250 0x7f197800b9c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1988009fc0 tx=0x7f1978005830 comp rx=0 tx=0).stop 2024-04-25T11:00:17.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.482+0000 7f19970b1640 1 -- 172.21.15.69:0/3919886321 shutdown_connections 2024-04-25T11:00:17.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.482+0000 7f19970b1640 1 --2- 172.21.15.69:0/3919886321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1978005250 0x7f197800b9c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.482+0000 7f19970b1640 1 -- 172.21.15.69:0/3919886321 >> 172.21.15.69:0/3919886321 conn(0x7f199009b7d0 msgr2=0x7f19900b7040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:17.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.482+0000 7f19970b1640 1 -- 172.21.15.69:0/3919886321 shutdown_connections 2024-04-25T11:00:17.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.482+0000 7f19970b1640 1 -- 172.21.15.69:0/3919886321 wait complete. 2024-04-25T11:00:17.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.484+0000 7f19970b1640 1 Processor -- start 2024-04-25T11:00:17.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.484+0000 7f19970b1640 1 -- start start 2024-04-25T11:00:17.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.486+0000 7f19970b1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1978005250 0x7f19900cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:17.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.486+0000 7f19970b1640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f199015f680 0x7f19901ebaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:17.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.486+0000 7f19970b1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19901ec030 0x7f199009d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:17.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.486+0000 7f19970b1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f19900cf610 con 0x7f1978005250 2024-04-25T11:00:17.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.486+0000 7f19970b1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f19900b3810 con 0x7f19901ec030 2024-04-25T11:00:17.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.486+0000 7f19970b1640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f19900cf790 con 0x7f199015f680 2024-04-25T11:00:17.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.486+0000 7f1994e4d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1978005250 0x7f19900cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:17.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.486+0000 7f1994e4d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1978005250 0x7f19900cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51114/0 (socket says 172.21.15.69:51114) 2024-04-25T11:00:17.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.486+0000 7f1994e4d640 1 -- 172.21.15.69:0/1030458384 learned_addr learned my addr 172.21.15.69:0/1030458384 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:17.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.486+0000 7f198ffff640 1 --2- 172.21.15.69:0/1030458384 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f199015f680 0x7f19901ebaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:17.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.486+0000 7f199564e640 1 --2- 172.21.15.69:0/1030458384 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19901ec030 0x7f199009d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:17.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.486+0000 7f1994e4d640 1 -- 172.21.15.69:0/1030458384 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f199015f680 msgr2=0x7f19901ebaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:17.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.486+0000 7f1994e4d640 1 --2- 172.21.15.69:0/1030458384 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f199015f680 0x7f19901ebaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.486+0000 7f1994e4d640 1 -- 172.21.15.69:0/1030458384 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19901ec030 msgr2=0x7f199009d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:17.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.486+0000 7f1994e4d640 1 --2- 172.21.15.69:0/1030458384 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19901ec030 0x7f199009d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.486+0000 7f1994e4d640 1 -- 172.21.15.69:0/1030458384 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19900ce620 con 0x7f1978005250 2024-04-25T11:00:17.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.486+0000 7f1994e4d640 1 --2- 172.21.15.69:0/1030458384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1978005250 0x7f19900cdbd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f198000e480 tx=0x7f198000e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:17.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.486+0000 7f198d7fa640 1 -- 172.21.15.69:0/1030458384 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19800052e0 con 0x7f1978005250 2024-04-25T11:00:17.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.486+0000 7f198d7fa640 1 -- 172.21.15.69:0/1030458384 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1980007020 con 0x7f1978005250 2024-04-25T11:00:17.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.486+0000 7f19970b1640 1 -- 172.21.15.69:0/1030458384 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19901ea9f0 con 0x7f1978005250 2024-04-25T11:00:17.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.486+0000 7f198d7fa640 1 -- 172.21.15.69:0/1030458384 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19800071f0 con 0x7f1978005250 2024-04-25T11:00:17.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.486+0000 7f19970b1640 1 -- 172.21.15.69:0/1030458384 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19900cd0f0 con 0x7f1978005250 2024-04-25T11:00:17.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.487+0000 7f19970b1640 1 -- 172.21.15.69:0/1030458384 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1990164010 con 0x7f1978005250 2024-04-25T11:00:17.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.488+0000 7f198d7fa640 1 -- 172.21.15.69:0/1030458384 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f198001a020 con 0x7f1978005250 2024-04-25T11:00:17.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.488+0000 7f198d7fa640 1 --2- 172.21.15.69:0/1030458384 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f197c0744b0 0x7f197c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:17.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.489+0000 7f198d7fa640 1 -- 172.21.15.69:0/1030458384 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f1980094740 con 0x7f1978005250 2024-04-25T11:00:17.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.489+0000 7f198ffff640 1 --2- 172.21.15.69:0/1030458384 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f197c0744b0 0x7f197c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:17.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.489+0000 7f198ffff640 1 --2- 172.21.15.69:0/1030458384 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f197c0744b0 0x7f197c0769c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f19901f9e90 tx=0x7f1984009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:17.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.493+0000 7f198d7fa640 1 -- 172.21.15.69:0/1030458384 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1990164010 con 0x7f1978005250 2024-04-25T11:00:17.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.702+0000 7f19970b1640 1 -- 172.21.15.69:0/1030458384 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1990050740 con 0x7f1978005250 2024-04-25T11:00:17.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.703+0000 7f198d7fa640 1 -- 172.21.15.69:0/1030458384 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) ==== 53+0+73 (secure 0 0 0) 0x7f198005ef60 con 0x7f1978005250 2024-04-25T11:00:17.703 INFO:teuthology.orchestra.run.smithi069.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:00:17.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.704+0000 7f19970b1640 1 -- 172.21.15.69:0/1030458384 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f197c0744b0 msgr2=0x7f197c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:17.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.704+0000 7f19970b1640 1 --2- 172.21.15.69:0/1030458384 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f197c0744b0 0x7f197c0769c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f19901f9e90 tx=0x7f1984009040 comp rx=0 tx=0).stop 2024-04-25T11:00:17.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.704+0000 7f19970b1640 1 -- 172.21.15.69:0/1030458384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1978005250 msgr2=0x7f19900cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:17.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.704+0000 7f19970b1640 1 --2- 172.21.15.69:0/1030458384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1978005250 0x7f19900cdbd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f198000e480 tx=0x7f198000e940 comp rx=0 tx=0).stop 2024-04-25T11:00:17.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.705+0000 7f19970b1640 1 -- 172.21.15.69:0/1030458384 shutdown_connections 2024-04-25T11:00:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.705+0000 7f19970b1640 1 --2- 172.21.15.69:0/1030458384 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f197c0744b0 0x7f197c0769c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.705+0000 7f19970b1640 1 --2- 172.21.15.69:0/1030458384 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19901ec030 0x7f199009d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.705+0000 7f19970b1640 1 --2- 172.21.15.69:0/1030458384 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f199015f680 0x7f19901ebaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.705+0000 7f19970b1640 1 --2- 172.21.15.69:0/1030458384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1978005250 0x7f19900cdbd0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.705+0000 7f19970b1640 1 -- 172.21.15.69:0/1030458384 >> 172.21.15.69:0/1030458384 conn(0x7f199009b7d0 msgr2=0x7f19900b2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.705+0000 7f19970b1640 1 -- 172.21.15.69:0/1030458384 shutdown_connections 2024-04-25T11:00:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17.705+0000 7f19970b1640 1 -- 172.21.15.69:0/1030458384 wait complete. 2024-04-25T11:00:17.716 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:00:17.716 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:00:17.716 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi069.front.sepia.ceph.com 2024-04-25T11:00:17.717 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:00:17.717 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:00:17.717 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:00:17.717 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:00:17.717 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:00:17.717 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:00:17.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:00:17.717 DEBUG:teuthology.orchestra.run.smithi069:> ip addr 2024-04-25T11:00:17.732 INFO:teuthology.orchestra.run.smithi069.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:00:17.732 INFO:teuthology.orchestra.run.smithi069.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:00:17.732 INFO:teuthology.orchestra.run.smithi069.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:00:17.732 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:00:17.733 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 ::1/128 scope host 2024-04-25T11:00:17.733 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:00:17.733 INFO:teuthology.orchestra.run.smithi069.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:00:17.733 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:d8:d9:c2 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:00:17.733 INFO:teuthology.orchestra.run.smithi069.stdout: altname enp7s0f0 2024-04-25T11:00:17.733 INFO:teuthology.orchestra.run.smithi069.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:00:17.733 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:d8:d9:c3 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:00:17.733 INFO:teuthology.orchestra.run.smithi069.stdout: altname enp7s0f1 2024-04-25T11:00:17.733 INFO:teuthology.orchestra.run.smithi069.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:00:17.733 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:8f:d7:36 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:00:17.733 INFO:teuthology.orchestra.run.smithi069.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:00:17.733 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:8f:d7:37 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:00:17.733 INFO:teuthology.orchestra.run.smithi069.stdout: inet 172.21.15.69/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T11:00:17.734 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft 42698sec preferred_lft 42698sec 2024-04-25T11:00:17.734 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 fe80::adbc:4e47:e7f9:8987/64 scope link noprefixroute 2024-04-25T11:00:17.734 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:00:17.734 INFO:teuthology.orchestra.run.smithi069.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:00:17.734 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 1a:fd:43:9b:af:c0 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:00:17.734 INFO:teuthology.orchestra.run.smithi069.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:00:17.734 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:00:17.734 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 fe80::10bd:ccff:fe05:fc3e/64 scope link 2024-04-25T11:00:17.734 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:00:17.734 INFO:teuthology.orchestra.run.smithi069.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:00:17.734 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 1a:fd:43:9b:af:c0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:00:17.734 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 fe80::18fd:43ff:fe9b:afc0/64 scope link 2024-04-25T11:00:17.734 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:00:17.735 INFO:teuthology.orchestra.run.smithi069.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:00:17.735 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 7a:e5:0f:bd:16:3f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:00:17.735 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 fe80::78e5:fff:febd:163f/64 scope link 2024-04-25T11:00:17.735 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:00:17.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:00:17.735 DEBUG:teuthology.orchestra.run.smithi069:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:00:17.794 INFO:teuthology.orchestra.run.smithi069.stdout:1 2024-04-25T11:00:17.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:00:17.796 DEBUG:teuthology.orchestra.run.smithi069:> route 2024-04-25T11:00:17.852 INFO:teuthology.orchestra.run.smithi069.stdout:Kernel IP routing table 2024-04-25T11:00:17.852 INFO:teuthology.orchestra.run.smithi069.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:00:17.852 INFO:teuthology.orchestra.run.smithi069.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T11:00:17.852 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T11:00:17.852 INFO:teuthology.orchestra.run.smithi069.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:00:17.853 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:00:17.853 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-04-25T11:00:17.853 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T11:00:17.853 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T11:00:17.853 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T11:00:17.853 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-04-25T11:00:17.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:00:17.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:00:17.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:00:17.976 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2024-04-25T11:00:17.990 INFO:teuthology.orchestra.run.smithi069.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:00:17.990 INFO:teuthology.orchestra.run.smithi069.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:00:17.991 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:00:17.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:00:17.991 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set brx.0 up 2024-04-25T11:00:18.054 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:00:18.054 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:00:18.109 INFO:teuthology.orchestra.run.smithi069.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:00:18.109 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:00:18.109 DEBUG:teuthology.orchestra.run.smithi069:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:00:18.164 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:00:18.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:00:18.164 DEBUG:teuthology.orchestra.run.smithi069:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:00:18.217 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:00:18.249 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:00:18.256 INFO:teuthology.orchestra.run.smithi069.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:00:18.256 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:00:18.264 INFO:teuthology.orchestra.run.smithi069.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:00:18.264 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:00:18.264 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: resolved to: "172.21.15.69:3300,172.21.15.146:3300,172.21.15.146:3301" 2024-04-25T11:00:18.264 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: trying mount with new device syntax: 0@ed0c8fe4-1bbb-4705-be72-eeea5bef859e.cephfs=/ 2024-04-25T11:00:18.264 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.69:3300/172.21.15.146:3300/172.21.15.146:3301" will pass to kernel 2024-04-25T11:00:18.265 INFO:teuthology.orchestra.run.smithi069.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:00:18.265 INFO:teuthology.orchestra.run.smithi069.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:00:18.265 INFO:teuthology.orchestra.run.smithi069.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:00:18.265 INFO:teuthology.orchestra.run.smithi069.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:00:18.265 INFO:teuthology.orchestra.run.smithi069.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:00:18.267 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:00:18.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:00:18.267 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:00:18.331 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:00:18.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:00:18.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:00:18.431 INFO:teuthology.orchestra.run.smithi069.stdout:client4820 2024-04-25T11:00:18.431 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T11:00:18.432 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:00:18.447 DEBUG:teuthology.orchestra.run.smithi069:> stdin-killer -- python3 -c ' 2024-04-25T11:00:18.448 DEBUG:teuthology.orchestra.run.smithi069:> import os 2024-04-25T11:00:18.448 DEBUG:teuthology.orchestra.run.smithi069:> os.mkdir("/home/ubuntu/cephtest/mnt.0/testdir") 2024-04-25T11:00:18.448 DEBUG:teuthology.orchestra.run.smithi069:> fd = os.open("/home/ubuntu/cephtest/mnt.0/testdir", os.O_RDONLY) 2024-04-25T11:00:18.448 DEBUG:teuthology.orchestra.run.smithi069:> os.fchmod(fd, 0o777) 2024-04-25T11:00:18.448 DEBUG:teuthology.orchestra.run.smithi069:> os.fsync(fd) 2024-04-25T11:00:18.448 DEBUG:teuthology.orchestra.run.smithi069:> ' 2024-04-25T11:00:18.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:00:18.567 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:00:18.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.714+0000 7f68eb55f640 1 Processor -- start 2024-04-25T11:00:18.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.715+0000 7f68eb55f640 1 -- start start 2024-04-25T11:00:18.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.716+0000 7f68eb55f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f68e40b6a00 0x7f68e415d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:18.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.716+0000 7f68eb55f640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68e41611e0 con 0x7f68e415a2f0 2024-04-25T11:00:18.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.716+0000 7f68eb55f640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68e4160420 con 0x7f68e406d560 2024-04-25T11:00:18.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.716+0000 7f68eb55f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f68e4160d50 con 0x7f68e40b6a00 2024-04-25T11:00:18.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.716+0000 7f68e8afa640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f68e415a2f0 0x7f68e406ce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:35546/0 (socket says 172.21.15.69:35546) 2024-04-25T11:00:18.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.716+0000 7f68e8afa640 1 -- 172.21.15.69:0/1269506259 learned_addr learned my addr 172.21.15.69:0/1269506259 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:18.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.716+0000 7f68e92fb640 1 --2- 172.21.15.69:0/1269506259 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f68e40b6a00 0x7f68e415d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:18.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.716+0000 7f68dbfff640 1 -- 172.21.15.69:0/1269506259 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3945202385 0 0) 0x7f68e41611e0 con 0x7f68e415a2f0 2024-04-25T11:00:18.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.716+0000 7f68dbfff640 1 -- 172.21.15.69:0/1269506259 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f68d0003520 con 0x7f68e415a2f0 2024-04-25T11:00:18.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.717+0000 7f68dbfff640 1 -- 172.21.15.69:0/1269506259 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3321074166 0 0) 0x7f68e4160420 con 0x7f68e406d560 2024-04-25T11:00:18.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.717+0000 7f68dbfff640 1 -- 172.21.15.69:0/1269506259 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f68e41611e0 con 0x7f68e406d560 2024-04-25T11:00:18.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.717+0000 7f68dbfff640 1 -- 172.21.15.69:0/1269506259 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2140663894 0 0) 0x7f68d0003520 con 0x7f68e415a2f0 2024-04-25T11:00:18.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.717+0000 7f68dbfff640 1 -- 172.21.15.69:0/1269506259 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f68e40b6a00 msgr2=0x7f68e415d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:18.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.717+0000 7f68dbfff640 1 --2- 172.21.15.69:0/1269506259 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f68e40b6a00 0x7f68e415d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:18.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.717+0000 7f68dbfff640 1 -- 172.21.15.69:0/1269506259 >> v1:172.21.15.146:6789/0 conn(0x7f68e406d560 legacy=0x7f68e40b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:18.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.717+0000 7f68dbfff640 1 -- 172.21.15.69:0/1269506259 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e4160700 con 0x7f68e415a2f0 2024-04-25T11:00:18.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.717+0000 7f68dbfff640 1 -- 172.21.15.69:0/1269506259 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7f68d4001670 con 0x7f68e415a2f0 2024-04-25T11:00:18.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.717+0000 7f68dbfff640 1 -- 172.21.15.69:0/1269506259 >> v1:172.21.15.69:6789/0 conn(0x7f68e415a2f0 legacy=0x7f68e406ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:18.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.717+0000 7f68dbfff640 1 --2- 172.21.15.69:0/1269506259 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f68e40b6a00 0x7f68d0003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:18.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.717+0000 7f68dbfff640 1 --2- 172.21.15.69:0/1269506259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68d0004120 0x7f68d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:18.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.717+0000 7f68dbfff640 1 --2- 172.21.15.69:0/1269506259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f68d0004d40 0x7f68d0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:18.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.717+0000 7f68dbfff640 1 -- 172.21.15.69:0/1269506259 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f68e4160d50 con 0x7f68d0004120 2024-04-25T11:00:18.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.717+0000 7f68dbfff640 1 -- 172.21.15.69:0/1269506259 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f68d000bb80 con 0x7f68d0004d40 2024-04-25T11:00:18.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.717+0000 7f68dbfff640 1 -- 172.21.15.69:0/1269506259 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f68d000bd80 con 0x7f68e40b6a00 2024-04-25T11:00:18.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.717+0000 7f68e8afa640 1 --2- 172.21.15.69:0/1269506259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68d0004120 0x7f68d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:18.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.717+0000 7f68e92fb640 1 --2- 172.21.15.69:0/1269506259 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f68e40b6a00 0x7f68d0003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:18.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.717+0000 7f68e9afc640 1 --2- 172.21.15.69:0/1269506259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f68d0004d40 0x7f68d0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:18.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.717+0000 7f68e8afa640 1 -- 172.21.15.69:0/1269506259 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f68e40b6a00 msgr2=0x7f68d0003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:18.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.717+0000 7f68e8afa640 1 --2- 172.21.15.69:0/1269506259 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f68e40b6a00 0x7f68d0003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:18.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.717+0000 7f68e8afa640 1 -- 172.21.15.69:0/1269506259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f68d0004d40 msgr2=0x7f68d0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:18.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.717+0000 7f68e8afa640 1 --2- 172.21.15.69:0/1269506259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f68d0004d40 0x7f68d0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:18.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.717+0000 7f68e8afa640 1 -- 172.21.15.69:0/1269506259 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d0003520 con 0x7f68d0004120 2024-04-25T11:00:18.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.717+0000 7f68e9afc640 1 --2- 172.21.15.69:0/1269506259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f68d0004d40 0x7f68d0005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:18.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.718+0000 7f68e8afa640 1 --2- 172.21.15.69:0/1269506259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68d0004120 0x7f68d0004800 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f68d4007fc0 tx=0x7f68d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e98220b91660a72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:18.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.718+0000 7f68dbfff640 1 -- 172.21.15.69:0/1269506259 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d4007be0 con 0x7f68d0004120 2024-04-25T11:00:18.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.718+0000 7f68dbfff640 1 -- 172.21.15.69:0/1269506259 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68d4005af0 con 0x7f68d0004120 2024-04-25T11:00:18.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.718+0000 7f68dbfff640 1 -- 172.21.15.69:0/1269506259 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d4007070 con 0x7f68d0004120 2024-04-25T11:00:18.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.719+0000 7f68eb55f640 1 -- 172.21.15.69:0/1269506259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68d0004120 msgr2=0x7f68d0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:18.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.719+0000 7f68eb55f640 1 --2- 172.21.15.69:0/1269506259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68d0004120 0x7f68d0004800 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f68d4007fc0 tx=0x7f68d0005830 comp rx=0 tx=0).stop 2024-04-25T11:00:18.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.719+0000 7f68e9afc640 1 -- 172.21.15.69:0/1269506259 reap_dead start 2024-04-25T11:00:18.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.719+0000 7f68eb55f640 1 -- 172.21.15.69:0/1269506259 shutdown_connections 2024-04-25T11:00:18.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.719+0000 7f68eb55f640 1 -- 172.21.15.69:0/1269506259 >> 172.21.15.69:0/1269506259 conn(0x7f68e40d1ae0 msgr2=0x7f68e40d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:18.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.719+0000 7f68eb55f640 1 -- 172.21.15.69:0/1269506259 shutdown_connections 2024-04-25T11:00:18.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.719+0000 7f68eb55f640 1 -- 172.21.15.69:0/1269506259 wait complete. 2024-04-25T11:00:18.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.721+0000 7f68eb55f640 1 Processor -- start 2024-04-25T11:00:18.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.721+0000 7f68eb55f640 1 -- start start 2024-04-25T11:00:18.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.722+0000 7f68eb55f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f68d0004120 0x7f68e41554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:18.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.722+0000 7f68eb55f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68e41f0180 0x7f68e41f0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:18.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.722+0000 7f68eb55f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f68e41f0ad0 0x7f68e41f9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:18.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.722+0000 7f68eb55f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f68e40ce580 con 0x7f68e41f0180 2024-04-25T11:00:18.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.722+0000 7f68eb55f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f68e415a700 con 0x7f68d0004120 2024-04-25T11:00:18.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.722+0000 7f68eb55f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f68e40ce700 con 0x7f68e41f0ad0 2024-04-25T11:00:18.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.722+0000 7f68e8afa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68e41f0180 0x7f68e41f0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:18.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.722+0000 7f68e8afa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68e41f0180 0x7f68e41f0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51138/0 (socket says 172.21.15.69:51138) 2024-04-25T11:00:18.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.722+0000 7f68e8afa640 1 -- 172.21.15.69:0/749677222 learned_addr learned my addr 172.21.15.69:0/749677222 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:18.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.722+0000 7f68e92fb640 1 --2- 172.21.15.69:0/749677222 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f68d0004120 0x7f68e41554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:18.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.722+0000 7f68e8afa640 1 -- 172.21.15.69:0/749677222 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f68e41f0ad0 msgr2=0x7f68e41f9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:18.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.722+0000 7f68e8afa640 1 --2- 172.21.15.69:0/749677222 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f68e41f0ad0 0x7f68e41f9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:18.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.722+0000 7f68e8afa640 1 -- 172.21.15.69:0/749677222 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f68d0004120 msgr2=0x7f68e41554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:18.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.722+0000 7f68e8afa640 1 --2- 172.21.15.69:0/749677222 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f68d0004120 0x7f68e41554f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:18.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.722+0000 7f68e8afa640 1 -- 172.21.15.69:0/749677222 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e40cad90 con 0x7f68e41f0180 2024-04-25T11:00:18.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.722+0000 7f68e8afa640 1 --2- 172.21.15.69:0/749677222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68e41f0180 0x7f68e41f0590 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f68d402acb0 tx=0x7f68e41608d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:18.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.723+0000 7f68d9ffb640 1 -- 172.21.15.69:0/749677222 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d4001670 con 0x7f68e41f0180 2024-04-25T11:00:18.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.723+0000 7f68d9ffb640 1 -- 172.21.15.69:0/749677222 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68d4042070 con 0x7f68e41f0180 2024-04-25T11:00:18.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.723+0000 7f68eb55f640 1 -- 172.21.15.69:0/749677222 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68e4158050 con 0x7f68e41f0180 2024-04-25T11:00:18.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.723+0000 7f68d9ffb640 1 -- 172.21.15.69:0/749677222 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d4007290 con 0x7f68e41f0180 2024-04-25T11:00:18.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.723+0000 7f68eb55f640 1 -- 172.21.15.69:0/749677222 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68e415f530 con 0x7f68e41f0180 2024-04-25T11:00:18.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.724+0000 7f68eb55f640 1 -- 172.21.15.69:0/749677222 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68e40705a0 con 0x7f68e41f0180 2024-04-25T11:00:18.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.724+0000 7f68d9ffb640 1 -- 172.21.15.69:0/749677222 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f68d4007430 con 0x7f68e41f0180 2024-04-25T11:00:18.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.725+0000 7f68d9ffb640 1 --2- 172.21.15.69:0/749677222 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f68c80744b0 0x7f68c80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:18.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.725+0000 7f68d9ffb640 1 -- 172.21.15.69:0/749677222 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f68d40b52c0 con 0x7f68e41f0180 2024-04-25T11:00:18.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.725+0000 7f68e92fb640 1 --2- 172.21.15.69:0/749677222 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f68c80744b0 0x7f68c80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:18.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.725+0000 7f68e92fb640 1 --2- 172.21.15.69:0/749677222 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f68c80744b0 0x7f68c80769c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f68e4070570 tx=0x7f68e0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:18.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.730+0000 7f68d9ffb640 1 -- 172.21.15.69:0/749677222 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f68d4005ac0 con 0x7f68e41f0180 2024-04-25T11:00:18.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.938+0000 7f68eb55f640 1 -- 172.21.15.69:0/749677222 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68e415efe0 con 0x7f68e41f0180 2024-04-25T11:00:18.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.939+0000 7f68d9ffb640 1 -- 172.21.15.69:0/749677222 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5177 (secure 0 0 0) 0x7f68d407fa80 con 0x7f68e41f0180 2024-04-25T11:00:18.939 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:00:18.939 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2223802951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":2223802951},{"type":"v1","addr":"172.21.15.146:6835","nonce":2223802951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4867,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/8243043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":8243043},{"type":"v1","addr":"172.21.15.146:6834","nonce":8243043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5031,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/3766456547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3766456547},{"type":"v1","addr":"172.21.15.69:6836","nonce":3766456547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:00:06.608766+0000","modified":"2024-04-25T11:00:09.621738+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5034},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5034":{"gid":5034,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6837/3153221129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":3153221129},{"type":"v1","addr":"172.21.15.69:6837","nonce":3153221129}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5034,"qdb_cluster":[5034]},"id":2}]} 2024-04-25T11:00:18.940 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 15 2024-04-25T11:00:18.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.941+0000 7f68eb55f640 1 -- 172.21.15.69:0/749677222 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f68c80744b0 msgr2=0x7f68c80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:18.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.941+0000 7f68eb55f640 1 --2- 172.21.15.69:0/749677222 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f68c80744b0 0x7f68c80769c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f68e4070570 tx=0x7f68e0008040 comp rx=0 tx=0).stop 2024-04-25T11:00:18.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.941+0000 7f68eb55f640 1 -- 172.21.15.69:0/749677222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68e41f0180 msgr2=0x7f68e41f0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:18.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.941+0000 7f68eb55f640 1 --2- 172.21.15.69:0/749677222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68e41f0180 0x7f68e41f0590 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f68d402acb0 tx=0x7f68e41608d0 comp rx=0 tx=0).stop 2024-04-25T11:00:18.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.941+0000 7f68eb55f640 1 -- 172.21.15.69:0/749677222 shutdown_connections 2024-04-25T11:00:18.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.941+0000 7f68eb55f640 1 --2- 172.21.15.69:0/749677222 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f68c80744b0 0x7f68c80769c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:18.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.941+0000 7f68eb55f640 1 --2- 172.21.15.69:0/749677222 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f68e41f0ad0 0x7f68e41f9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:18.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.941+0000 7f68eb55f640 1 --2- 172.21.15.69:0/749677222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68e41f0180 0x7f68e41f0590 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:18.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.941+0000 7f68eb55f640 1 --2- 172.21.15.69:0/749677222 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f68d0004120 0x7f68e41554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:18.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.941+0000 7f68eb55f640 1 -- 172.21.15.69:0/749677222 >> 172.21.15.69:0/749677222 conn(0x7f68e40d1ae0 msgr2=0x7f68e40b5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:18.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.941+0000 7f68eb55f640 1 -- 172.21.15.69:0/749677222 shutdown_connections 2024-04-25T11:00:18.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:18.941+0000 7f68eb55f640 1 -- 172.21.15.69:0/749677222 wait complete. 2024-04-25T11:00:18.953 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T11:00:19.083 INFO:teuthology.orchestra.run.smithi069.stdout:{"message":"","return_code":0} 2024-04-25T11:00:19.083 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T11:00:19.084 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:00:19.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.190+0000 7f3f26b7d640 1 Processor -- start 2024-04-25T11:00:19.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.190+0000 7f3f26b7d640 1 -- start start 2024-04-25T11:00:19.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.192+0000 7f3f26b7d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f200cfd40 0x7f3f200b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:19.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.192+0000 7f3f26b7d640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f3f200b7f50 0x7f3f200ce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:19.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.192+0000 7f3f26b7d640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f200d1c10 con 0x7f3f200b3e00 2024-04-25T11:00:19.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.192+0000 7f3f26b7d640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f3f200d27e0 con 0x7f3f200b7f50 2024-04-25T11:00:19.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.192+0000 7f3f26b7d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3f200d1eb0 con 0x7f3f200cfd40 2024-04-25T11:00:19.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.192+0000 7f3f2511a640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f3f200b7f50 0x7f3f200ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:19.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.192+0000 7f3f1ffff640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f3f200b3e00 0x7f3f200b7840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:35558/0 (socket says 172.21.15.69:35558) 2024-04-25T11:00:19.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.192+0000 7f3f2511a640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f3f200b7f50 0x7f3f200ce110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51154/0 (socket says 172.21.15.69:51154) 2024-04-25T11:00:19.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.192+0000 7f3f1ffff640 1 -- 172.21.15.69:0/379682276 learned_addr learned my addr 172.21.15.69:0/379682276 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:19.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.192+0000 7f3f24919640 1 --2- 172.21.15.69:0/379682276 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f200cfd40 0x7f3f200b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:19.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.192+0000 7f3f2511a640 1 -- 172.21.15.69:0/379682276 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f200cfd40 msgr2=0x7f3f200b21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:19.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.192+0000 7f3f2511a640 1 --2- 172.21.15.69:0/379682276 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f200cfd40 0x7f3f200b21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:19.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.192+0000 7f3f2511a640 1 -- 172.21.15.69:0/379682276 >> v1:172.21.15.69:6789/0 conn(0x7f3f200b3e00 legacy=0x7f3f200b7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:19.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.192+0000 7f3f2511a640 1 -- 172.21.15.69:0/379682276 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f200d2190 con 0x7f3f200b7f50 2024-04-25T11:00:19.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.192+0000 7f3f1f7fe640 1 -- 172.21.15.69:0/379682276 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2358173617 0 0) 0x7f3f200d1c10 con 0x7f3f200b3e00 2024-04-25T11:00:19.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.193+0000 7f3f2511a640 1 --2- 172.21.15.69:0/379682276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f200b7f50 0x7f3f200ce110 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3f200d2960 tx=0x7f3f1802a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e748e174267346e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:19.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.193+0000 7f3f1f7fe640 1 -- 172.21.15.69:0/379682276 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1802ae60 con 0x7f3f200b7f50 2024-04-25T11:00:19.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.193+0000 7f3f1f7fe640 1 -- 172.21.15.69:0/379682276 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f18007740 con 0x7f3f200b7f50 2024-04-25T11:00:19.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.193+0000 7f3f1f7fe640 1 -- 172.21.15.69:0/379682276 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f18007b10 con 0x7f3f200b7f50 2024-04-25T11:00:19.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.194+0000 7f3f26b7d640 1 -- 172.21.15.69:0/379682276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f200b7f50 msgr2=0x7f3f200ce110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:19.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.194+0000 7f3f26b7d640 1 --2- 172.21.15.69:0/379682276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f200b7f50 0x7f3f200ce110 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3f200d2960 tx=0x7f3f1802a640 comp rx=0 tx=0).stop 2024-04-25T11:00:19.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.194+0000 7f3f26b7d640 1 -- 172.21.15.69:0/379682276 shutdown_connections 2024-04-25T11:00:19.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.194+0000 7f3f26b7d640 1 --2- 172.21.15.69:0/379682276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f200b7f50 0x7f3f200ce110 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:19.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.194+0000 7f3f26b7d640 1 --2- 172.21.15.69:0/379682276 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f200cfd40 0x7f3f200b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:19.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.194+0000 7f3f26b7d640 1 -- 172.21.15.69:0/379682276 >> 172.21.15.69:0/379682276 conn(0x7f3f2009b7d0 msgr2=0x7f3f2009bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:19.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.194+0000 7f3f26b7d640 1 -- 172.21.15.69:0/379682276 shutdown_connections 2024-04-25T11:00:19.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.194+0000 7f3f26b7d640 1 -- 172.21.15.69:0/379682276 wait complete. 2024-04-25T11:00:19.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.195+0000 7f3f26b7d640 1 Processor -- start 2024-04-25T11:00:19.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.196+0000 7f3f26b7d640 1 -- start start 2024-04-25T11:00:19.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.197+0000 7f3f26b7d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f200b3e00 0x7f3f200a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:19.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.197+0000 7f3f26b7d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f200cfd40 0x7f3f2006d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:19.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.197+0000 7f3f26b7d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f2006d9d0 0x7f3f200b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:19.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.197+0000 7f3f26b7d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3f200b4210 con 0x7f3f2006d9d0 2024-04-25T11:00:19.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.197+0000 7f3f26b7d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3f200ce650 con 0x7f3f200cfd40 2024-04-25T11:00:19.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.197+0000 7f3f26b7d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f3f200b4390 con 0x7f3f200b3e00 2024-04-25T11:00:19.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.197+0000 7f3f2511a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f2006d9d0 0x7f3f200b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:19.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.197+0000 7f3f2511a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f2006d9d0 0x7f3f200b07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51166/0 (socket says 172.21.15.69:51166) 2024-04-25T11:00:19.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.197+0000 7f3f2511a640 1 -- 172.21.15.69:0/3468290414 learned_addr learned my addr 172.21.15.69:0/3468290414 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:19.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.197+0000 7f3f24919640 1 --2- 172.21.15.69:0/3468290414 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f200b3e00 0x7f3f200a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:19.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.197+0000 7f3f1ffff640 1 --2- 172.21.15.69:0/3468290414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f200cfd40 0x7f3f2006d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:19.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.197+0000 7f3f2511a640 1 -- 172.21.15.69:0/3468290414 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f200b3e00 msgr2=0x7f3f200a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:19.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.197+0000 7f3f2511a640 1 --2- 172.21.15.69:0/3468290414 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f200b3e00 0x7f3f200a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:19.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.197+0000 7f3f2511a640 1 -- 172.21.15.69:0/3468290414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f200cfd40 msgr2=0x7f3f2006d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:19.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.197+0000 7f3f2511a640 1 --2- 172.21.15.69:0/3468290414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f200cfd40 0x7f3f2006d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:19.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.197+0000 7f3f2511a640 1 -- 172.21.15.69:0/3468290414 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f200afa30 con 0x7f3f2006d9d0 2024-04-25T11:00:19.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.197+0000 7f3f1ffff640 1 --2- 172.21.15.69:0/3468290414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f200cfd40 0x7f3f2006d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:19.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.198+0000 7f3f2511a640 1 --2- 172.21.15.69:0/3468290414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f2006d9d0 0x7f3f200b07f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3f1802ac30 tx=0x7f3f18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:19.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.198+0000 7f3f1d7fa640 1 -- 172.21.15.69:0/3468290414 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1803f070 con 0x7f3f2006d9d0 2024-04-25T11:00:19.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.198+0000 7f3f1d7fa640 1 -- 172.21.15.69:0/3468290414 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f18031d40 con 0x7f3f2006d9d0 2024-04-25T11:00:19.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.198+0000 7f3f1d7fa640 1 -- 172.21.15.69:0/3468290414 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f18009450 con 0x7f3f2006d9d0 2024-04-25T11:00:19.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.198+0000 7f3f26b7d640 1 -- 172.21.15.69:0/3468290414 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f200afd10 con 0x7f3f2006d9d0 2024-04-25T11:00:19.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.198+0000 7f3f26b7d640 1 -- 172.21.15.69:0/3468290414 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f201f67a0 con 0x7f3f2006d9d0 2024-04-25T11:00:19.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.199+0000 7f3f26b7d640 1 -- 172.21.15.69:0/3468290414 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f20050740 con 0x7f3f2006d9d0 2024-04-25T11:00:19.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.200+0000 7f3f1d7fa640 1 -- 172.21.15.69:0/3468290414 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3f1803b410 con 0x7f3f2006d9d0 2024-04-25T11:00:19.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.200+0000 7f3f1d7fa640 1 --2- 172.21.15.69:0/3468290414 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3f0c0744e0 0x7f3f0c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:19.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.200+0000 7f3f1d7fa640 1 -- 172.21.15.69:0/3468290414 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f3f180b6440 con 0x7f3f2006d9d0 2024-04-25T11:00:19.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.200+0000 7f3f24919640 1 --2- 172.21.15.69:0/3468290414 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3f0c0744e0 0x7f3f0c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:19.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.201+0000 7f3f24919640 1 --2- 172.21.15.69:0/3468290414 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3f0c0744e0 0x7f3f0c0769f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3f100079a0 tx=0x7f3f10008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:19.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.205+0000 7f3f1d7fa640 1 -- 172.21.15.69:0/3468290414 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3f18080c60 con 0x7f3f2006d9d0 2024-04-25T11:00:19.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.420+0000 7f3f26b7d640 1 -- 172.21.15.69:0/3468290414 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f200a07c0 con 0x7f3f2006d9d0 2024-04-25T11:00:19.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.420+0000 7f3f1d7fa640 1 -- 172.21.15.69:0/3468290414 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5177 (secure 0 0 0) 0x7f3f18080560 con 0x7f3f2006d9d0 2024-04-25T11:00:19.421 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:00:19.421 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2223802951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":2223802951},{"type":"v1","addr":"172.21.15.146:6835","nonce":2223802951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4867,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/8243043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":8243043},{"type":"v1","addr":"172.21.15.146:6834","nonce":8243043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5031,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/3766456547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3766456547},{"type":"v1","addr":"172.21.15.69:6836","nonce":3766456547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:00:06.608766+0000","modified":"2024-04-25T11:00:09.621738+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5034},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5034":{"gid":5034,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.69:6837/3153221129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":3153221129},{"type":"v1","addr":"172.21.15.69:6837","nonce":3153221129}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5034,"qdb_cluster":[5034]},"id":2}]} 2024-04-25T11:00:19.421 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 15 2024-04-25T11:00:19.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.423+0000 7f3f26b7d640 1 -- 172.21.15.69:0/3468290414 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3f0c0744e0 msgr2=0x7f3f0c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:19.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.423+0000 7f3f26b7d640 1 --2- 172.21.15.69:0/3468290414 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3f0c0744e0 0x7f3f0c0769f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3f100079a0 tx=0x7f3f10008040 comp rx=0 tx=0).stop 2024-04-25T11:00:19.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.423+0000 7f3f26b7d640 1 -- 172.21.15.69:0/3468290414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f2006d9d0 msgr2=0x7f3f200b07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:19.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.423+0000 7f3f26b7d640 1 --2- 172.21.15.69:0/3468290414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f2006d9d0 0x7f3f200b07f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3f1802ac30 tx=0x7f3f18004a40 comp rx=0 tx=0).stop 2024-04-25T11:00:19.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.423+0000 7f3f26b7d640 1 -- 172.21.15.69:0/3468290414 shutdown_connections 2024-04-25T11:00:19.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.423+0000 7f3f26b7d640 1 --2- 172.21.15.69:0/3468290414 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f3f0c0744e0 0x7f3f0c0769f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:19.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.423+0000 7f3f26b7d640 1 --2- 172.21.15.69:0/3468290414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f2006d9d0 0x7f3f200b07f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:19.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.423+0000 7f3f26b7d640 1 --2- 172.21.15.69:0/3468290414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f200cfd40 0x7f3f2006d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:19.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.423+0000 7f3f26b7d640 1 --2- 172.21.15.69:0/3468290414 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f200b3e00 0x7f3f200a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:19.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.423+0000 7f3f26b7d640 1 -- 172.21.15.69:0/3468290414 >> 172.21.15.69:0/3468290414 conn(0x7f3f2009b7d0 msgr2=0x7f3f2009bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:19.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.423+0000 7f3f26b7d640 1 -- 172.21.15.69:0/3468290414 shutdown_connections 2024-04-25T11:00:19.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19.423+0000 7f3f26b7d640 1 -- 172.21.15.69:0/3468290414 wait complete. 2024-04-25T11:00:19.435 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_log_pause 1 2024-04-25T11:00:19.554 INFO:teuthology.orchestra.run.smithi069.stdout:{"success":"mds_log_pause = '' "} 2024-04-25T11:00:19.554 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_log_pause = '' " } 2024-04-25T11:00:19.555 DEBUG:teuthology.orchestra.run.smithi069:> sudo stdin-killer -- python3 -c ' 2024-04-25T11:00:19.555 DEBUG:teuthology.orchestra.run.smithi069:> import os 2024-04-25T11:00:19.555 DEBUG:teuthology.orchestra.run.smithi069:> os.chdir("/home/ubuntu/cephtest/mnt.0/testdir") 2024-04-25T11:00:19.555 DEBUG:teuthology.orchestra.run.smithi069:> os.setgid(65534) 2024-04-25T11:00:19.555 DEBUG:teuthology.orchestra.run.smithi069:> os.setuid(65534) 2024-04-25T11:00:19.555 DEBUG:teuthology.orchestra.run.smithi069:> fd = os.open("testfile", os.O_CREAT | os.O_RDWR, 0o644) 2024-04-25T11:00:19.555 DEBUG:teuthology.orchestra.run.smithi069:> os.fchmod(fd, 0o640) 2024-04-25T11:00:19.555 DEBUG:teuthology.orchestra.run.smithi069:> ' 2024-04-25T11:00:19.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:00:19.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:00:19.639 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo chmod 600 /home/ubuntu/cephtest/mnt.0/testdir/testfile) 2024-04-25T11:00:29.682 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 2:0 2024-04-25T11:00:29.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.805+0000 7ff4b3b57640 1 Processor -- start 2024-04-25T11:00:29.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.805+0000 7ff4b3b57640 1 -- start start 2024-04-25T11:00:29.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.806+0000 7ff4b3b57640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7ff4ac0c53a0 0x7ff4ac1622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.806+0000 7ff4b3b57640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4ac071c80 con 0x7ff4ac06c5c0 2024-04-25T11:00:29.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.806+0000 7ff4b3b57640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4ac0cd670 con 0x7ff4ac15e940 2024-04-25T11:00:29.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.806+0000 7ff4b3b57640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7ff4ac0cc7a0 con 0x7ff4ac0c53a0 2024-04-25T11:00:29.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.806+0000 7ff4b18f3640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7ff4ac06c5c0 0x7ff4ac161be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:55720/0 (socket says 172.21.15.69:55720) 2024-04-25T11:00:29.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.806+0000 7ff4b18f3640 1 -- 172.21.15.69:0/2548263366 learned_addr learned my addr 172.21.15.69:0/2548263366 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:29.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.806+0000 7ff4b10f2640 1 --2- 172.21.15.69:0/2548263366 >> v2:172.21.15.69:3300/0 conn(0x7ff4ac0c53a0 0x7ff4ac1622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.807+0000 7ff4b10f2640 1 -- 172.21.15.69:0/2548263366 >> v1:172.21.15.146:6789/0 conn(0x7ff4ac15e940 legacy=0x7ff4ac15ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:29.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.807+0000 7ff4b10f2640 1 -- 172.21.15.69:0/2548263366 >> v1:172.21.15.69:6789/0 conn(0x7ff4ac06c5c0 legacy=0x7ff4ac161be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:29.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.807+0000 7ff4b10f2640 1 -- 172.21.15.69:0/2548263366 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ac0cd950 con 0x7ff4ac0c53a0 2024-04-25T11:00:29.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.807+0000 7ff4b08f1640 1 -- 172.21.15.69:0/2548263366 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4142923164 0 0) 0x7ff4ac071c80 con 0x7ff4ac06c5c0 2024-04-25T11:00:29.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.807+0000 7ff4b10f2640 1 --2- 172.21.15.69:0/2548263366 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4ac0c53a0 0x7ff4ac1622f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff49c0097a0 tx=0x7ff49c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c3b61f48c61e13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:29.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.808+0000 7ff4b08f1640 1 -- 172.21.15.69:0/2548263366 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49c037070 con 0x7ff4ac0c53a0 2024-04-25T11:00:29.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.808+0000 7ff4b08f1640 1 -- 172.21.15.69:0/2548263366 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff49c032440 con 0x7ff4ac0c53a0 2024-04-25T11:00:29.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.808+0000 7ff4b08f1640 1 -- 172.21.15.69:0/2548263366 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49c02ae60 con 0x7ff4ac0c53a0 2024-04-25T11:00:29.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.808+0000 7ff4b3b57640 1 -- 172.21.15.69:0/2548263366 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4ac0c53a0 msgr2=0x7ff4ac1622f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:29.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.808+0000 7ff4b3b57640 1 --2- 172.21.15.69:0/2548263366 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4ac0c53a0 0x7ff4ac1622f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff49c0097a0 tx=0x7ff49c02a110 comp rx=0 tx=0).stop 2024-04-25T11:00:29.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.808+0000 7ff4b3b57640 1 -- 172.21.15.69:0/2548263366 shutdown_connections 2024-04-25T11:00:29.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.808+0000 7ff4b3b57640 1 --2- 172.21.15.69:0/2548263366 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4ac0c53a0 0x7ff4ac1622f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.808+0000 7ff4b3b57640 1 -- 172.21.15.69:0/2548263366 >> 172.21.15.69:0/2548263366 conn(0x7ff4ac0ce790 msgr2=0x7ff4ac0ceba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:29.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.809+0000 7ff4b3b57640 1 -- 172.21.15.69:0/2548263366 shutdown_connections 2024-04-25T11:00:29.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.809+0000 7ff4b3b57640 1 -- 172.21.15.69:0/2548263366 wait complete. 2024-04-25T11:00:29.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.810+0000 7ff4b3b57640 1 Processor -- start 2024-04-25T11:00:29.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.810+0000 7ff4b3b57640 1 -- start start 2024-04-25T11:00:29.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.812+0000 7ff4b3b57640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4ac06c5c0 0x7ff4ac0cb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.812+0000 7ff4b3b57640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff4ac0c53a0 0x7ff4ac1593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.812+0000 7ff4b3b57640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4ac15e940 0x7ff4ac0c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.812+0000 7ff4b3b57640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac0b7870 con 0x7ff4ac06c5c0 2024-04-25T11:00:29.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.812+0000 7ff4b3b57640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac0cdcc0 con 0x7ff4ac15e940 2024-04-25T11:00:29.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.812+0000 7ff4b3b57640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff4ac071f60 con 0x7ff4ac0c53a0 2024-04-25T11:00:29.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.812+0000 7ff4b18f3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4ac06c5c0 0x7ff4ac0cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.812+0000 7ff4b18f3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4ac06c5c0 0x7ff4ac0cb710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49494/0 (socket says 172.21.15.69:49494) 2024-04-25T11:00:29.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.812+0000 7ff4b18f3640 1 -- 172.21.15.69:0/891189886 learned_addr learned my addr 172.21.15.69:0/891189886 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:29.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.812+0000 7ff4b10f2640 1 --2- 172.21.15.69:0/891189886 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff4ac0c53a0 0x7ff4ac1593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.812+0000 7ff4b20f4640 1 --2- 172.21.15.69:0/891189886 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4ac15e940 0x7ff4ac0c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.812+0000 7ff4b18f3640 1 -- 172.21.15.69:0/891189886 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff4ac0c53a0 msgr2=0x7ff4ac1593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:29.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.812+0000 7ff4b18f3640 1 --2- 172.21.15.69:0/891189886 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff4ac0c53a0 0x7ff4ac1593f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.812+0000 7ff4b18f3640 1 -- 172.21.15.69:0/891189886 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4ac15e940 msgr2=0x7ff4ac0c7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:29.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.812+0000 7ff4b18f3640 1 --2- 172.21.15.69:0/891189886 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4ac15e940 0x7ff4ac0c7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:29.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.812+0000 7ff4b18f3640 1 -- 172.21.15.69:0/891189886 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ac0c58e0 con 0x7ff4ac06c5c0 2024-04-25T11:00:29.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.812+0000 7ff4b18f3640 1 --2- 172.21.15.69:0/891189886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4ac06c5c0 0x7ff4ac0cb710 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff4a800c3c0 tx=0x7ff4a800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:29.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.812+0000 7ff4a27fc640 1 -- 172.21.15.69:0/891189886 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a80074e0 con 0x7ff4ac06c5c0 2024-04-25T11:00:29.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.812+0000 7ff4a27fc640 1 -- 172.21.15.69:0/891189886 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4a80047e0 con 0x7ff4ac06c5c0 2024-04-25T11:00:29.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.812+0000 7ff4a27fc640 1 -- 172.21.15.69:0/891189886 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8004bd0 con 0x7ff4ac06c5c0 2024-04-25T11:00:29.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.812+0000 7ff4b3b57640 1 -- 172.21.15.69:0/891189886 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4ac0ca940 con 0x7ff4ac06c5c0 2024-04-25T11:00:29.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.812+0000 7ff4b3b57640 1 -- 172.21.15.69:0/891189886 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4ac0c6e10 con 0x7ff4ac06c5c0 2024-04-25T11:00:29.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.813+0000 7ff4b3b57640 1 -- 172.21.15.69:0/891189886 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4ac15f580 con 0x7ff4ac06c5c0 2024-04-25T11:00:29.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.814+0000 7ff4a27fc640 1 -- 172.21.15.69:0/891189886 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff4a8011070 con 0x7ff4ac06c5c0 2024-04-25T11:00:29.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.814+0000 7ff4a27fc640 1 --2- 172.21.15.69:0/891189886 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff4900744b0 0x7ff4900769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:29.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.815+0000 7ff4a27fc640 1 -- 172.21.15.69:0/891189886 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7ff4a8094e20 con 0x7ff4ac06c5c0 2024-04-25T11:00:29.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.814+0000 7ff4b10f2640 1 --2- 172.21.15.69:0/891189886 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff4900744b0 0x7ff4900769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:29.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.815+0000 7ff4b10f2640 1 --2- 172.21.15.69:0/891189886 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff4900744b0 0x7ff4900769c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff49c034000 tx=0x7ff49c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:29.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:29.819+0000 7ff4a27fc640 1 -- 172.21.15.69:0/891189886 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff4ac15f580 con 0x7ff4ac06c5c0 2024-04-25T11:00:30.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.065+0000 7ff4b3b57640 1 -- 172.21.15.69:0/891189886 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "2:0"} v 0) -- 0x7ff4ac050740 con 0x7ff4ac06c5c0 2024-04-25T11:00:30.066 INFO:tasks.ceph.mon.a.smithi069.stderr:2024-04-25T11:00:30.066+0000 7f5831b98640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:00:30.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.067+0000 7ff4a27fc640 1 -- 172.21.15.69:0/891189886 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "2:0"}]=0 failed mds gid 5034 v16) ==== 97+0+0 (secure 0 0 0) 0x7ff4a805f5b0 con 0x7ff4ac06c5c0 2024-04-25T11:00:30.068 INFO:teuthology.orchestra.run.smithi069.stderr:failed mds gid 5034 2024-04-25T11:00:30.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.069+0000 7ff4b3b57640 1 -- 172.21.15.69:0/891189886 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff4900744b0 msgr2=0x7ff4900769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:30.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.069+0000 7ff4b3b57640 1 --2- 172.21.15.69:0/891189886 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff4900744b0 0x7ff4900769c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff49c034000 tx=0x7ff49c02e000 comp rx=0 tx=0).stop 2024-04-25T11:00:30.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.069+0000 7ff4b3b57640 1 -- 172.21.15.69:0/891189886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4ac06c5c0 msgr2=0x7ff4ac0cb710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:30.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.069+0000 7ff4b3b57640 1 --2- 172.21.15.69:0/891189886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4ac06c5c0 0x7ff4ac0cb710 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff4a800c3c0 tx=0x7ff4a800c880 comp rx=0 tx=0).stop 2024-04-25T11:00:30.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.069+0000 7ff4b3b57640 1 -- 172.21.15.69:0/891189886 shutdown_connections 2024-04-25T11:00:30.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.069+0000 7ff4b3b57640 1 --2- 172.21.15.69:0/891189886 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff4900744b0 0x7ff4900769c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:30.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.069+0000 7ff4b3b57640 1 --2- 172.21.15.69:0/891189886 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4ac15e940 0x7ff4ac0c7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:30.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.069+0000 7ff4b3b57640 1 --2- 172.21.15.69:0/891189886 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff4ac0c53a0 0x7ff4ac1593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:30.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.069+0000 7ff4b3b57640 1 --2- 172.21.15.69:0/891189886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4ac06c5c0 0x7ff4ac0cb710 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:30.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.069+0000 7ff4b3b57640 1 -- 172.21.15.69:0/891189886 >> 172.21.15.69:0/891189886 conn(0x7ff4ac0ce790 msgr2=0x7ff4ac06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:30.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.070+0000 7ff4b3b57640 1 -- 172.21.15.69:0/891189886 shutdown_connections 2024-04-25T11:00:30.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.070+0000 7ff4b3b57640 1 -- 172.21.15.69:0/891189886 wait complete. 2024-04-25T11:00:30.072 INFO:tasks.ceph.mds.a.smithi069.stderr: -3024> 2024-04-25T11:00:12.073+0000 7fca9755b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:00:30.085 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:00:30.128 INFO:tasks.ceph.mds.a.smithi069.stdout:starting mds.a at 2024-04-25T11:00:30.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.194+0000 7fa981538640 1 Processor -- start 2024-04-25T11:00:30.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.194+0000 7fa981538640 1 -- start start 2024-04-25T11:00:30.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.195+0000 7fa981538640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa97c0d1210 0x7fa97c0d5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:30.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.195+0000 7fa981538640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa97c0d1850 con 0x7fa97c0d0e00 2024-04-25T11:00:30.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.196+0000 7fa981538640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa97c160de0 con 0x7fa97c06ccd0 2024-04-25T11:00:30.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.196+0000 7fa981538640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fa97c0d1620 con 0x7fa97c0d1210 2024-04-25T11:00:30.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.196+0000 7fa97affd640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fa97c0d0e00 0x7fa97c06c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:55730/0 (socket says 172.21.15.69:55730) 2024-04-25T11:00:30.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.196+0000 7fa97affd640 1 -- 172.21.15.69:0/1292643394 learned_addr learned my addr 172.21.15.69:0/1292643394 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:30.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.196+0000 7fa97b7fe640 1 --2- 172.21.15.69:0/1292643394 >> v2:172.21.15.146:3300/0 conn(0x7fa97c0d1210 0x7fa97c0d5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:30.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.196+0000 7fa979ffb640 1 -- 172.21.15.69:0/1292643394 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3595875356 0 0) 0x7fa97c0d1850 con 0x7fa97c0d0e00 2024-04-25T11:00:30.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.196+0000 7fa979ffb640 1 -- 172.21.15.69:0/1292643394 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa964003520 con 0x7fa97c0d0e00 2024-04-25T11:00:30.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.196+0000 7fa979ffb640 1 -- 172.21.15.69:0/1292643394 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2167155379 0 0) 0x7fa97c160de0 con 0x7fa97c06ccd0 2024-04-25T11:00:30.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.196+0000 7fa979ffb640 1 -- 172.21.15.69:0/1292643394 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa97c0d1850 con 0x7fa97c06ccd0 2024-04-25T11:00:30.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.196+0000 7fa97b7fe640 1 -- 172.21.15.69:0/1292643394 >> v1:172.21.15.146:6789/0 conn(0x7fa97c06ccd0 legacy=0x7fa97c0d4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:30.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.196+0000 7fa97b7fe640 1 -- 172.21.15.69:0/1292643394 >> v1:172.21.15.69:6789/0 conn(0x7fa97c0d0e00 legacy=0x7fa97c06c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:30.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.196+0000 7fa979ffb640 1 -- 172.21.15.69:0/1292643394 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2248629954 0 0) 0x7fa964003520 con 0x7fa97c0d0e00 2024-04-25T11:00:30.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.196+0000 7fa97b7fe640 1 -- 172.21.15.69:0/1292643394 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa97c1610c0 con 0x7fa97c0d1210 2024-04-25T11:00:30.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.197+0000 7fa97b7fe640 1 --2- 172.21.15.69:0/1292643394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa97c0d1210 0x7fa97c0d5160 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa9740097a0 tx=0x7fa97402a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50d7bcbdc3f32272 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:30.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.197+0000 7fa979ffb640 1 -- 172.21.15.69:0/1292643394 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974037070 con 0x7fa97c0d1210 2024-04-25T11:00:30.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.197+0000 7fa979ffb640 1 -- 172.21.15.69:0/1292643394 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa97402abc0 con 0x7fa97c0d1210 2024-04-25T11:00:30.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.197+0000 7fa979ffb640 1 -- 172.21.15.69:0/1292643394 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c0019f0 con 0x7fa97c0d1210 2024-04-25T11:00:30.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.198+0000 7fa981538640 1 -- 172.21.15.69:0/1292643394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa97c0d1210 msgr2=0x7fa97c0d5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:30.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.198+0000 7fa981538640 1 --2- 172.21.15.69:0/1292643394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa97c0d1210 0x7fa97c0d5160 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa9740097a0 tx=0x7fa97402a130 comp rx=0 tx=0).stop 2024-04-25T11:00:30.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.198+0000 7fa981538640 1 -- 172.21.15.69:0/1292643394 shutdown_connections 2024-04-25T11:00:30.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.198+0000 7fa981538640 1 --2- 172.21.15.69:0/1292643394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa97c0d1210 0x7fa97c0d5160 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:30.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.198+0000 7fa981538640 1 -- 172.21.15.69:0/1292643394 >> 172.21.15.69:0/1292643394 conn(0x7fa97c0d5d90 msgr2=0x7fa97c0d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:30.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.198+0000 7fa981538640 1 -- 172.21.15.69:0/1292643394 shutdown_connections 2024-04-25T11:00:30.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.198+0000 7fa981538640 1 -- 172.21.15.69:0/1292643394 wait complete. 2024-04-25T11:00:30.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.200+0000 7fa981538640 1 Processor -- start 2024-04-25T11:00:30.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.200+0000 7fa981538640 1 -- start start 2024-04-25T11:00:30.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.201+0000 7fa981538640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa97c0d0e00 0x7fa97c0b70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:30.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.201+0000 7fa981538640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa97c0d1210 0x7fa97c0b75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:30.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.201+0000 7fa981538640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa97c154b10 0x7fa97c0ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:30.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.201+0000 7fa981538640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c0b5320 con 0x7fa97c0d1210 2024-04-25T11:00:30.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.201+0000 7fa981538640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c160ad0 con 0x7fa97c0d0e00 2024-04-25T11:00:30.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.201+0000 7fa981538640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fa97c164e00 con 0x7fa97c154b10 2024-04-25T11:00:30.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.201+0000 7fa97a7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa97c0d1210 0x7fa97c0b75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:30.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.201+0000 7fa97a7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa97c0d1210 0x7fa97c0b75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49512/0 (socket says 172.21.15.69:49512) 2024-04-25T11:00:30.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.201+0000 7fa97a7fc640 1 -- 172.21.15.69:0/3640931055 learned_addr learned my addr 172.21.15.69:0/3640931055 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:30.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.201+0000 7fa97affd640 1 --2- 172.21.15.69:0/3640931055 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa97c0d0e00 0x7fa97c0b70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:30.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.201+0000 7fa97b7fe640 1 --2- 172.21.15.69:0/3640931055 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa97c154b10 0x7fa97c0ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:30.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.202+0000 7fa97a7fc640 1 -- 172.21.15.69:0/3640931055 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa97c154b10 msgr2=0x7fa97c0ca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:30.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.202+0000 7fa97a7fc640 1 --2- 172.21.15.69:0/3640931055 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa97c154b10 0x7fa97c0ca6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:30.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.202+0000 7fa97a7fc640 1 -- 172.21.15.69:0/3640931055 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa97c0d0e00 msgr2=0x7fa97c0b70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:30.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.202+0000 7fa97a7fc640 1 --2- 172.21.15.69:0/3640931055 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa97c0d0e00 0x7fa97c0b70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:30.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.202+0000 7fa97a7fc640 1 -- 172.21.15.69:0/3640931055 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa97c0cd3b0 con 0x7fa97c0d1210 2024-04-25T11:00:30.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.202+0000 7fa97a7fc640 1 --2- 172.21.15.69:0/3640931055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa97c0d1210 0x7fa97c0b75f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa97000c430 tx=0x7fa97000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:30.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.202+0000 7fa9637fe640 1 -- 172.21.15.69:0/3640931055 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa970017070 con 0x7fa97c0d1210 2024-04-25T11:00:30.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.202+0000 7fa9637fe640 1 -- 172.21.15.69:0/3640931055 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9700052f0 con 0x7fa97c0d1210 2024-04-25T11:00:30.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.202+0000 7fa981538640 1 -- 172.21.15.69:0/3640931055 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa97c0cd690 con 0x7fa97c0d1210 2024-04-25T11:00:30.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.202+0000 7fa9637fe640 1 -- 172.21.15.69:0/3640931055 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa970003830 con 0x7fa97c0d1210 2024-04-25T11:00:30.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.202+0000 7fa981538640 1 -- 172.21.15.69:0/3640931055 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa97c0cdc50 con 0x7fa97c0d1210 2024-04-25T11:00:30.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.203+0000 7fa981538640 1 -- 172.21.15.69:0/3640931055 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa97c0cd120 con 0x7fa97c0d1210 2024-04-25T11:00:30.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.204+0000 7fa9637fe640 1 -- 172.21.15.69:0/3640931055 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa97001a070 con 0x7fa97c0d1210 2024-04-25T11:00:30.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.204+0000 7fa9637fe640 1 --2- 172.21.15.69:0/3640931055 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa9680744b0 0x7fa9680769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:30.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.204+0000 7fa97affd640 1 --2- 172.21.15.69:0/3640931055 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa9680744b0 0x7fa9680769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:30.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.204+0000 7fa9637fe640 1 -- 172.21.15.69:0/3640931055 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7fa970094950 con 0x7fa97c0d1210 2024-04-25T11:00:30.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.205+0000 7fa97affd640 1 --2- 172.21.15.69:0/3640931055 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa9680744b0 0x7fa9680769c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa96c001b60 tx=0x7fa96c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:30.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.208+0000 7fa9637fe640 1 -- 172.21.15.69:0/3640931055 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa970021020 con 0x7fa97c0d1210 2024-04-25T11:00:30.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.425+0000 7fa981538640 1 -- 172.21.15.69:0/3640931055 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa97c050740 con 0x7fa97c0d1210 2024-04-25T11:00:30.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.426+0000 7fa9637fe640 1 -- 172.21.15.69:0/3640931055 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+4347 (secure 0 0 0) 0x7fa9700671e0 con 0x7fa97c0d1210 2024-04-25T11:00:30.427 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:00:30.427 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2223802951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":2223802951},{"type":"v1","addr":"172.21.15.146:6835","nonce":2223802951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4867,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/8243043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":8243043},{"type":"v1","addr":"172.21.15.146:6834","nonce":8243043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:00:06.608766+0000","modified":"2024-04-25T11:00:30.071855+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5031":{"gid":5031,"name":"c","rank":0,"incarnation":17,"state":"up:replay","state_seq":1,"addr":"172.21.15.69:6836/3766456547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3766456547},{"type":"v1","addr":"172.21.15.69:6836","nonce":3766456547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-25T11:00:30.427 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 17 2024-04-25T11:00:30.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.428+0000 7fa981538640 1 -- 172.21.15.69:0/3640931055 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa9680744b0 msgr2=0x7fa9680769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:30.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.428+0000 7fa981538640 1 --2- 172.21.15.69:0/3640931055 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa9680744b0 0x7fa9680769c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa96c001b60 tx=0x7fa96c009040 comp rx=0 tx=0).stop 2024-04-25T11:00:30.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.428+0000 7fa981538640 1 -- 172.21.15.69:0/3640931055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa97c0d1210 msgr2=0x7fa97c0b75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:30.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.428+0000 7fa981538640 1 --2- 172.21.15.69:0/3640931055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa97c0d1210 0x7fa97c0b75f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa97000c430 tx=0x7fa97000c8f0 comp rx=0 tx=0).stop 2024-04-25T11:00:30.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.428+0000 7fa981538640 1 -- 172.21.15.69:0/3640931055 shutdown_connections 2024-04-25T11:00:30.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.428+0000 7fa981538640 1 --2- 172.21.15.69:0/3640931055 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fa9680744b0 0x7fa9680769c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:30.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.428+0000 7fa981538640 1 --2- 172.21.15.69:0/3640931055 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa97c154b10 0x7fa97c0ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:30.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.428+0000 7fa981538640 1 --2- 172.21.15.69:0/3640931055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa97c0d1210 0x7fa97c0b75f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:30.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.428+0000 7fa981538640 1 --2- 172.21.15.69:0/3640931055 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa97c0d0e00 0x7fa97c0b70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:30.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.428+0000 7fa981538640 1 -- 172.21.15.69:0/3640931055 >> 172.21.15.69:0/3640931055 conn(0x7fa97c0d5d90 msgr2=0x7fa97c163010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:30.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.428+0000 7fa981538640 1 -- 172.21.15.69:0/3640931055 shutdown_connections 2024-04-25T11:00:30.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:30.428+0000 7fa981538640 1 -- 172.21.15.69:0/3640931055 wait complete. 2024-04-25T11:00:30.845 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:00:06.608766+0000', 'modified': '2024-04-25T11:00:30.071855+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 32, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5031}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5031': {'gid': 5031, 'name': 'c', 'rank': 0, 'incarnation': 17, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.69:6836/3766456547', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6834', 'nonce': 3766456547}, {'type': 'v1', 'addr': '172.21.15.69:6836', 'nonce': 3766456547}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T11:00:30.845 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5031:up:replay 2024-04-25T11:00:31.846 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:00:31.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.965+0000 7f4480c27640 1 Processor -- start 2024-04-25T11:00:31.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.965+0000 7f4480c27640 1 -- start start 2024-04-25T11:00:31.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.966+0000 7f4480c27640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f447c0cf5e0 0x7f447c06d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:31.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.966+0000 7f4480c27640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f447c06f7a0 0x7f447c15ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:31.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.966+0000 7f4480c27640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f447c06c660 con 0x7f447c06d640 2024-04-25T11:00:31.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.966+0000 7f4480c27640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f447c06fef0 con 0x7f447c0cf5e0 2024-04-25T11:00:31.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.966+0000 7f4480c27640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f447c06cc70 con 0x7f447c06f7a0 2024-04-25T11:00:31.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.967+0000 7f447a59c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f447c0cf5e0 0x7f447c06d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:31.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.967+0000 7f447a59c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f447c0cf5e0 0x7f447c06d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49514/0 (socket says 172.21.15.69:49514) 2024-04-25T11:00:31.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.967+0000 7f447a59c640 1 -- 172.21.15.69:0/3092895533 learned_addr learned my addr 172.21.15.69:0/3092895533 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:31.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.967+0000 7f447a59c640 1 -- 172.21.15.69:0/3092895533 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f447c06f7a0 msgr2=0x7f447c15ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:31.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.967+0000 7f447a59c640 1 --2- 172.21.15.69:0/3092895533 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f447c06f7a0 0x7f447c15ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:31.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.967+0000 7f447a59c640 1 -- 172.21.15.69:0/3092895533 >> v1:172.21.15.146:6789/0 conn(0x7f447c06d640 legacy=0x7f447c06f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:31.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.967+0000 7f447a59c640 1 -- 172.21.15.69:0/3092895533 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447c0cfbc0 con 0x7f447c0cf5e0 2024-04-25T11:00:31.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.968+0000 7f447a59c640 1 --2- 172.21.15.69:0/3092895533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f447c0cf5e0 0x7f447c06d100 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f447c072820 tx=0x7f446c02a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb2e7222e00f9bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:31.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.968+0000 7f4479d9b640 1 -- 172.21.15.69:0/3092895533 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c037070 con 0x7f447c0cf5e0 2024-04-25T11:00:31.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.968+0000 7f4479d9b640 1 -- 172.21.15.69:0/3092895533 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f446c032440 con 0x7f447c0cf5e0 2024-04-25T11:00:31.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.968+0000 7f4479d9b640 1 -- 172.21.15.69:0/3092895533 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c032810 con 0x7f447c0cf5e0 2024-04-25T11:00:31.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.968+0000 7f4480c27640 1 -- 172.21.15.69:0/3092895533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f447c0cf5e0 msgr2=0x7f447c06d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:31.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.968+0000 7f4480c27640 1 --2- 172.21.15.69:0/3092895533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f447c0cf5e0 0x7f447c06d100 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f447c072820 tx=0x7f446c02a160 comp rx=0 tx=0).stop 2024-04-25T11:00:31.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.968+0000 7f4480c27640 1 -- 172.21.15.69:0/3092895533 shutdown_connections 2024-04-25T11:00:31.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.968+0000 7f4480c27640 1 --2- 172.21.15.69:0/3092895533 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f447c06f7a0 0x7f447c15ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:31.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.968+0000 7f4480c27640 1 --2- 172.21.15.69:0/3092895533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f447c0cf5e0 0x7f447c06d100 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:31.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.968+0000 7f4480c27640 1 -- 172.21.15.69:0/3092895533 >> 172.21.15.69:0/3092895533 conn(0x7f447c0d1ae0 msgr2=0x7f447c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:31.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.968+0000 7f4480c27640 1 -- 172.21.15.69:0/3092895533 shutdown_connections 2024-04-25T11:00:31.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.969+0000 7f4480c27640 1 -- 172.21.15.69:0/3092895533 wait complete. 2024-04-25T11:00:31.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.970+0000 7f4480c27640 1 Processor -- start 2024-04-25T11:00:31.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.970+0000 7f4480c27640 1 -- start start 2024-04-25T11:00:31.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.972+0000 7f4480c27640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f447c06d640 0x7f447c15d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:31.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.972+0000 7f4480c27640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f447c0cf5e0 0x7f447c1434d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:31.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.972+0000 7f4480c27640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f447c15d8e0 0x7f447c146950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:31.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.972+0000 7f4480c27640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f447c0d0020 con 0x7f447c06d640 2024-04-25T11:00:31.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.972+0000 7f4480c27640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f447c0cfea0 con 0x7f447c0cf5e0 2024-04-25T11:00:31.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.972+0000 7f4480c27640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f447c15f210 con 0x7f447c15d8e0 2024-04-25T11:00:31.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.972+0000 7f447a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f447c06d640 0x7f447c15d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:31.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.972+0000 7f447a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f447c06d640 0x7f447c15d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49516/0 (socket says 172.21.15.69:49516) 2024-04-25T11:00:31.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.972+0000 7f447a59c640 1 -- 172.21.15.69:0/2710354872 learned_addr learned my addr 172.21.15.69:0/2710354872 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:31.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.972+0000 7f447ad9d640 1 --2- 172.21.15.69:0/2710354872 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f447c15d8e0 0x7f447c146950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:31.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.972+0000 7f4473fff640 1 --2- 172.21.15.69:0/2710354872 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f447c0cf5e0 0x7f447c1434d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:31.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.972+0000 7f447a59c640 1 -- 172.21.15.69:0/2710354872 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f447c15d8e0 msgr2=0x7f447c146950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:31.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.972+0000 7f447a59c640 1 --2- 172.21.15.69:0/2710354872 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f447c15d8e0 0x7f447c146950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:31.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.972+0000 7f447a59c640 1 -- 172.21.15.69:0/2710354872 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f447c0cf5e0 msgr2=0x7f447c1434d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:31.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.972+0000 7f447a59c640 1 --2- 172.21.15.69:0/2710354872 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f447c0cf5e0 0x7f447c1434d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:31.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.972+0000 7f447a59c640 1 -- 172.21.15.69:0/2710354872 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447c144200 con 0x7f447c06d640 2024-04-25T11:00:31.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.972+0000 7f447a59c640 1 --2- 172.21.15.69:0/2710354872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f447c06d640 0x7f447c15d3a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f446c000c00 tx=0x7f446c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:31.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.973+0000 7f4472ffd640 1 -- 172.21.15.69:0/2710354872 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c041070 con 0x7f447c06d640 2024-04-25T11:00:31.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.973+0000 7f4472ffd640 1 -- 172.21.15.69:0/2710354872 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f446c03c870 con 0x7f447c06d640 2024-04-25T11:00:31.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.973+0000 7f4480c27640 1 -- 172.21.15.69:0/2710354872 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f447c1415c0 con 0x7f447c06d640 2024-04-25T11:00:31.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.973+0000 7f4472ffd640 1 -- 172.21.15.69:0/2710354872 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c037040 con 0x7f447c06d640 2024-04-25T11:00:31.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.973+0000 7f4480c27640 1 -- 172.21.15.69:0/2710354872 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f447c141b80 con 0x7f447c06d640 2024-04-25T11:00:31.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.974+0000 7f4480c27640 1 -- 172.21.15.69:0/2710354872 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f447c070760 con 0x7f447c06d640 2024-04-25T11:00:31.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.974+0000 7f4472ffd640 1 -- 172.21.15.69:0/2710354872 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f446c044070 con 0x7f447c06d640 2024-04-25T11:00:31.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.975+0000 7f4472ffd640 1 --2- 172.21.15.69:0/2710354872 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f445c0744e0 0x7f445c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:31.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.975+0000 7f4472ffd640 1 -- 172.21.15.69:0/2710354872 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7f446c0b5dd0 con 0x7f447c06d640 2024-04-25T11:00:31.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.975+0000 7f4473fff640 1 --2- 172.21.15.69:0/2710354872 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f445c0744e0 0x7f445c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:31.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.975+0000 7f4473fff640 1 --2- 172.21.15.69:0/2710354872 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f445c0744e0 0x7f445c0769f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f447c1f7b30 tx=0x7f4464008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:31.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:31.979+0000 7f4472ffd640 1 -- 172.21.15.69:0/2710354872 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f446c0805a0 con 0x7f447c06d640 2024-04-25T11:00:32.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:32.193+0000 7f4480c27640 1 -- 172.21.15.69:0/2710354872 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f447c162740 con 0x7f447c06d640 2024-04-25T11:00:32.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:32.194+0000 7f4472ffd640 1 -- 172.21.15.69:0/2710354872 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+5173 (secure 0 0 0) 0x7f446c07fea0 con 0x7f447c06d640 2024-04-25T11:00:32.195 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:00:32.195 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2223802951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":2223802951},{"type":"v1","addr":"172.21.15.146:6835","nonce":2223802951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4867,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/8243043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":8243043},{"type":"v1","addr":"172.21.15.146:6834","nonce":8243043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5250,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1639802532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1639802532},{"type":"v1","addr":"172.21.15.69:6837","nonce":1639802532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:00:06.608766+0000","modified":"2024-04-25T11:00:31.078862+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5031":{"gid":5031,"name":"c","rank":0,"incarnation":17,"state":"up:rejoin","state_seq":10,"addr":"172.21.15.69:6836/3766456547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3766456547},{"type":"v1","addr":"172.21.15.69:6836","nonce":3766456547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-25T11:00:32.195 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 19 2024-04-25T11:00:32.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:32.196+0000 7f4480c27640 1 -- 172.21.15.69:0/2710354872 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f445c0744e0 msgr2=0x7f445c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:32.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:32.196+0000 7f4480c27640 1 --2- 172.21.15.69:0/2710354872 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f445c0744e0 0x7f445c0769f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f447c1f7b30 tx=0x7f4464008040 comp rx=0 tx=0).stop 2024-04-25T11:00:32.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:32.196+0000 7f4480c27640 1 -- 172.21.15.69:0/2710354872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f447c06d640 msgr2=0x7f447c15d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:32.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:32.196+0000 7f4480c27640 1 --2- 172.21.15.69:0/2710354872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f447c06d640 0x7f447c15d3a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f446c000c00 tx=0x7f446c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:00:32.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:32.196+0000 7f4480c27640 1 -- 172.21.15.69:0/2710354872 shutdown_connections 2024-04-25T11:00:32.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:32.196+0000 7f4480c27640 1 --2- 172.21.15.69:0/2710354872 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f445c0744e0 0x7f445c0769f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:32.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:32.196+0000 7f4480c27640 1 --2- 172.21.15.69:0/2710354872 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f447c15d8e0 0x7f447c146950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:32.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:32.196+0000 7f4480c27640 1 --2- 172.21.15.69:0/2710354872 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f447c0cf5e0 0x7f447c1434d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:32.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:32.196+0000 7f4480c27640 1 --2- 172.21.15.69:0/2710354872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f447c06d640 0x7f447c15d3a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:32.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:32.196+0000 7f4480c27640 1 -- 172.21.15.69:0/2710354872 >> 172.21.15.69:0/2710354872 conn(0x7f447c0d1ae0 msgr2=0x7f447c154e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:32.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:32.196+0000 7f4480c27640 1 -- 172.21.15.69:0/2710354872 shutdown_connections 2024-04-25T11:00:32.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:32.196+0000 7f4480c27640 1 -- 172.21.15.69:0/2710354872 wait complete. 2024-04-25T11:00:32.610 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:00:06.608766+0000', 'modified': '2024-04-25T11:00:31.078862+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 32, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5031}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5031': {'gid': 5031, 'name': 'c', 'rank': 0, 'incarnation': 17, 'state': 'up:rejoin', 'state_seq': 10, 'addr': '172.21.15.69:6836/3766456547', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6834', 'nonce': 3766456547}, {'type': 'v1', 'addr': '172.21.15.69:6836', 'nonce': 3766456547}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T11:00:32.610 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5031:up:rejoin 2024-04-25T11:00:33.611 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:00:33.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.731+0000 7f7fe4701640 1 Processor -- start 2024-04-25T11:00:33.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.731+0000 7f7fe4701640 1 -- start start 2024-04-25T11:00:33.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.733+0000 7f7fe4701640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7fdc15f420 0x7f7fdc15e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:33.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.733+0000 7f7fe4701640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7fdc0b2d20 0x7f7fdc0b3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:33.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.733+0000 7f7fe4701640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fdc0b3670 0x7f7fdc161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:33.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.733+0000 7f7fe4701640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f7fdc0d2df0 con 0x7f7fdc0b2d20 2024-04-25T11:00:33.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.733+0000 7f7fe4701640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f7fdc0d2220 con 0x7f7fdc15f420 2024-04-25T11:00:33.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.733+0000 7f7fe4701640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7fdc0d24c0 con 0x7f7fdc0b3670 2024-04-25T11:00:33.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.733+0000 7f7fe1c9c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7fdc0b2d20 0x7f7fdc0b3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:33.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.733+0000 7f7fe1c9c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7fdc0b2d20 0x7f7fdc0b3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49530/0 (socket says 172.21.15.69:49530) 2024-04-25T11:00:33.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.733+0000 7f7fe1c9c640 1 -- 172.21.15.69:0/2176519528 learned_addr learned my addr 172.21.15.69:0/2176519528 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:33.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.733+0000 7f7fe249d640 1 --2- 172.21.15.69:0/2176519528 >> v2:172.21.15.146:3300/0 conn(0x7f7fdc15f420 0x7f7fdc15e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:33.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.733+0000 7f7fe2c9e640 1 --2- 172.21.15.69:0/2176519528 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fdc0b3670 0x7f7fdc161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:33.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.733+0000 7f7fe1c9c640 1 -- 172.21.15.69:0/2176519528 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fdc0b3670 msgr2=0x7f7fdc161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:33.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.733+0000 7f7fe1c9c640 1 --2- 172.21.15.69:0/2176519528 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fdc0b3670 0x7f7fdc161df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:33.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.733+0000 7f7fe1c9c640 1 -- 172.21.15.69:0/2176519528 >> v2:172.21.15.146:3300/0 conn(0x7f7fdc15f420 msgr2=0x7f7fdc15e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:33.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.733+0000 7f7fe1c9c640 1 --2- 172.21.15.69:0/2176519528 >> v2:172.21.15.146:3300/0 conn(0x7f7fdc15f420 0x7f7fdc15e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:33.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.733+0000 7f7fe1c9c640 1 -- 172.21.15.69:0/2176519528 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fdc0d27a0 con 0x7f7fdc0b2d20 2024-04-25T11:00:33.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.733+0000 7f7fe249d640 1 --2- 172.21.15.69:0/2176519528 >> v2:172.21.15.146:3300/0 conn(0x7f7fdc15f420 0x7f7fdc15e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:00:33.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.734+0000 7f7fe1c9c640 1 --2- 172.21.15.69:0/2176519528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fdc0b2d20 0x7f7fdc0b3130 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc0c5cf0 tx=0x7f7fcc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a01efe88289894 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:33.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.734+0000 7f7fe149b640 1 -- 172.21.15.69:0/2176519528 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc037070 con 0x7f7fdc0b2d20 2024-04-25T11:00:33.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.734+0000 7f7fe149b640 1 -- 172.21.15.69:0/2176519528 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fcc032440 con 0x7f7fdc0b2d20 2024-04-25T11:00:33.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.734+0000 7f7fe149b640 1 -- 172.21.15.69:0/2176519528 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc032810 con 0x7f7fdc0b2d20 2024-04-25T11:00:33.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.735+0000 7f7fe4701640 1 -- 172.21.15.69:0/2176519528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fdc0b2d20 msgr2=0x7f7fdc0b3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:33.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.735+0000 7f7fe4701640 1 --2- 172.21.15.69:0/2176519528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fdc0b2d20 0x7f7fdc0b3130 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc0c5cf0 tx=0x7f7fcc02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:00:33.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.735+0000 7f7fe4701640 1 -- 172.21.15.69:0/2176519528 shutdown_connections 2024-04-25T11:00:33.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.735+0000 7f7fe4701640 1 --2- 172.21.15.69:0/2176519528 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fdc0b3670 0x7f7fdc161df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:33.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.735+0000 7f7fe4701640 1 --2- 172.21.15.69:0/2176519528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fdc0b2d20 0x7f7fdc0b3130 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:33.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.735+0000 7f7fe4701640 1 --2- 172.21.15.69:0/2176519528 >> v2:172.21.15.146:3300/0 conn(0x7f7fdc15f420 0x7f7fdc15e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:33.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.735+0000 7f7fe4701640 1 -- 172.21.15.69:0/2176519528 >> 172.21.15.69:0/2176519528 conn(0x7f7fdc09b7d0 msgr2=0x7f7fdc0b7040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:33.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.735+0000 7f7fe4701640 1 -- 172.21.15.69:0/2176519528 shutdown_connections 2024-04-25T11:00:33.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.735+0000 7f7fe4701640 1 -- 172.21.15.69:0/2176519528 wait complete. 2024-04-25T11:00:33.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.737+0000 7f7fe4701640 1 Processor -- start 2024-04-25T11:00:33.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.737+0000 7f7fe4701640 1 -- start start 2024-04-25T11:00:33.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.738+0000 7f7fe4701640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fdc0b2d20 0x7f7fdc0c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:33.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.738+0000 7f7fe4701640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fdc0b3670 0x7f7fdc1f4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:33.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.738+0000 7f7fe4701640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fdc15f420 0x7f7fdc159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:33.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.738+0000 7f7fe4701640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc0b7750 con 0x7f7fdc0b2d20 2024-04-25T11:00:33.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.738+0000 7f7fe4701640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc0b3a80 con 0x7f7fdc15f420 2024-04-25T11:00:33.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.738+0000 7f7fe4701640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7fdc0b78d0 con 0x7f7fdc0b3670 2024-04-25T11:00:33.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.738+0000 7f7fe249d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fdc0b2d20 0x7f7fdc0c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:33.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.738+0000 7f7fe249d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fdc0b2d20 0x7f7fdc0c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49546/0 (socket says 172.21.15.69:49546) 2024-04-25T11:00:33.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.738+0000 7f7fe249d640 1 -- 172.21.15.69:0/1421995867 learned_addr learned my addr 172.21.15.69:0/1421995867 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:33.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.738+0000 7f7fe2c9e640 1 --2- 172.21.15.69:0/1421995867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fdc15f420 0x7f7fdc159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:33.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.738+0000 7f7fe1c9c640 1 --2- 172.21.15.69:0/1421995867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fdc0b3670 0x7f7fdc1f4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:33.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.738+0000 7f7fe249d640 1 -- 172.21.15.69:0/1421995867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fdc0b3670 msgr2=0x7f7fdc1f4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:33.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.738+0000 7f7fe249d640 1 --2- 172.21.15.69:0/1421995867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fdc0b3670 0x7f7fdc1f4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:33.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.738+0000 7f7fe249d640 1 -- 172.21.15.69:0/1421995867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fdc15f420 msgr2=0x7f7fdc159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:33.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.738+0000 7f7fe249d640 1 --2- 172.21.15.69:0/1421995867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fdc15f420 0x7f7fdc159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:33.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.739+0000 7f7fe249d640 1 -- 172.21.15.69:0/1421995867 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fdc1f5140 con 0x7f7fdc0b2d20 2024-04-25T11:00:33.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.739+0000 7f7fe249d640 1 --2- 172.21.15.69:0/1421995867 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fdc0b2d20 0x7f7fdc0c8aa0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800cde0 tx=0x7f7fd800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:33.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.739+0000 7f7fd2ffd640 1 -- 172.21.15.69:0/1421995867 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd800f680 con 0x7f7fdc0b2d20 2024-04-25T11:00:33.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.739+0000 7f7fd2ffd640 1 -- 172.21.15.69:0/1421995867 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fd800f820 con 0x7f7fdc0b2d20 2024-04-25T11:00:33.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.739+0000 7f7fe4701640 1 -- 172.21.15.69:0/1421995867 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fdc158260 con 0x7f7fdc0b2d20 2024-04-25T11:00:33.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.739+0000 7f7fd2ffd640 1 -- 172.21.15.69:0/1421995867 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd800fc10 con 0x7f7fdc0b2d20 2024-04-25T11:00:33.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.739+0000 7f7fe4701640 1 -- 172.21.15.69:0/1421995867 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fdc0ca7e0 con 0x7f7fdc0b2d20 2024-04-25T11:00:33.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.740+0000 7f7fe4701640 1 -- 172.21.15.69:0/1421995867 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fdc050740 con 0x7f7fdc0b2d20 2024-04-25T11:00:33.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.741+0000 7f7fd2ffd640 1 -- 172.21.15.69:0/1421995867 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7fd8018400 con 0x7f7fdc0b2d20 2024-04-25T11:00:33.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.741+0000 7f7fd2ffd640 1 --2- 172.21.15.69:0/1421995867 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f7fc00744b0 0x7f7fc00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:33.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.741+0000 7f7fd2ffd640 1 -- 172.21.15.69:0/1421995867 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7f7fd8096150 con 0x7f7fdc0b2d20 2024-04-25T11:00:33.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.741+0000 7f7fe1c9c640 1 --2- 172.21.15.69:0/1421995867 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f7fc00744b0 0x7f7fc00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:33.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.742+0000 7f7fe1c9c640 1 --2- 172.21.15.69:0/1421995867 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f7fc00744b0 0x7f7fc00769c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc034000 tx=0x7f7fcc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:33.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.746+0000 7f7fd2ffd640 1 -- 172.21.15.69:0/1421995867 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7fdc1f5140 con 0x7f7fdc0b2d20 2024-04-25T11:00:33.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.959+0000 7f7fe4701640 1 -- 172.21.15.69:0/1421995867 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7fdc0b4910 con 0x7f7fdc0b2d20 2024-04-25T11:00:33.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.960+0000 7f7fd2ffd640 1 -- 172.21.15.69:0/1421995867 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5179 (secure 0 0 0) 0x7f7fd8014030 con 0x7f7fdc0b2d20 2024-04-25T11:00:33.961 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:00:33.961 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2223802951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":2223802951},{"type":"v1","addr":"172.21.15.146:6835","nonce":2223802951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4867,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/8243043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":8243043},{"type":"v1","addr":"172.21.15.146:6834","nonce":8243043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5250,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1639802532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1639802532},{"type":"v1","addr":"172.21.15.69:6837","nonce":1639802532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:00:06.608766+0000","modified":"2024-04-25T11:00:32.081581+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5031":{"gid":5031,"name":"c","rank":0,"incarnation":17,"state":"up:clientreplay","state_seq":11,"addr":"172.21.15.69:6836/3766456547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3766456547},{"type":"v1","addr":"172.21.15.69:6836","nonce":3766456547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-25T11:00:33.961 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 20 2024-04-25T11:00:33.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.962+0000 7f7fe4701640 1 -- 172.21.15.69:0/1421995867 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f7fc00744b0 msgr2=0x7f7fc00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:33.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.962+0000 7f7fe4701640 1 --2- 172.21.15.69:0/1421995867 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f7fc00744b0 0x7f7fc00769c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc034000 tx=0x7f7fcc02e000 comp rx=0 tx=0).stop 2024-04-25T11:00:33.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.962+0000 7f7fe4701640 1 -- 172.21.15.69:0/1421995867 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fdc0b2d20 msgr2=0x7f7fdc0c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:33.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.962+0000 7f7fe4701640 1 --2- 172.21.15.69:0/1421995867 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fdc0b2d20 0x7f7fdc0c8aa0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800cde0 tx=0x7f7fd800a650 comp rx=0 tx=0).stop 2024-04-25T11:00:33.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.962+0000 7f7fe4701640 1 -- 172.21.15.69:0/1421995867 shutdown_connections 2024-04-25T11:00:33.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.962+0000 7f7fe4701640 1 --2- 172.21.15.69:0/1421995867 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f7fc00744b0 0x7f7fc00769c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:33.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.962+0000 7f7fe4701640 1 --2- 172.21.15.69:0/1421995867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fdc15f420 0x7f7fdc159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:33.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.962+0000 7f7fe4701640 1 --2- 172.21.15.69:0/1421995867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fdc0b3670 0x7f7fdc1f4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:33.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.962+0000 7f7fe4701640 1 --2- 172.21.15.69:0/1421995867 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fdc0b2d20 0x7f7fdc0c8aa0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:33.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.962+0000 7f7fe4701640 1 -- 172.21.15.69:0/1421995867 >> 172.21.15.69:0/1421995867 conn(0x7f7fdc09b7d0 msgr2=0x7f7fdc0b0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:33.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.962+0000 7f7fe4701640 1 -- 172.21.15.69:0/1421995867 shutdown_connections 2024-04-25T11:00:33.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:33.962+0000 7f7fe4701640 1 -- 172.21.15.69:0/1421995867 wait complete. 2024-04-25T11:00:34.376 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:00:06.608766+0000', 'modified': '2024-04-25T11:00:32.081581+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 32, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5031}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5031': {'gid': 5031, 'name': 'c', 'rank': 0, 'incarnation': 17, 'state': 'up:clientreplay', 'state_seq': 11, 'addr': '172.21.15.69:6836/3766456547', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6834', 'nonce': 3766456547}, {'type': 'v1', 'addr': '172.21.15.69:6836', 'nonce': 3766456547}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T11:00:34.376 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5031:up:clientreplay 2024-04-25T11:00:35.378 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:00:35.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.498+0000 7fb543670640 1 Processor -- start 2024-04-25T11:00:35.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.498+0000 7fb543670640 1 -- start start 2024-04-25T11:00:35.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.500+0000 7fb543670640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb53c0d0e00 0x7fb53c0d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:35.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.500+0000 7fb543670640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb53c06c5c0 0x7fb53c06c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:35.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.500+0000 7fb543670640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb53c06cf10 con 0x7fb53c0b7810 2024-04-25T11:00:35.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.500+0000 7fb543670640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fb53c0d5a60 con 0x7fb53c06c5c0 2024-04-25T11:00:35.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.500+0000 7fb543670640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb53c0d1850 con 0x7fb53c0d0e00 2024-04-25T11:00:35.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.500+0000 7fb540c0b640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb53c06c5c0 0x7fb53c06c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:35.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.500+0000 7fb541c0d640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fb53c0b7810 0x7fb53c0b7c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:39406/0 (socket says 172.21.15.69:39406) 2024-04-25T11:00:35.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.500+0000 7fb541c0d640 1 -- 172.21.15.69:0/824771374 learned_addr learned my addr 172.21.15.69:0/824771374 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:35.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.500+0000 7fb54140c640 1 --2- 172.21.15.69:0/824771374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb53c0d0e00 0x7fb53c0d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:35.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.500+0000 7fb540c0b640 1 -- 172.21.15.69:0/824771374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb53c0d0e00 msgr2=0x7fb53c0d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:35.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.500+0000 7fb540c0b640 1 --2- 172.21.15.69:0/824771374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb53c0d0e00 0x7fb53c0d1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:35.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.500+0000 7fb540c0b640 1 -- 172.21.15.69:0/824771374 >> v1:172.21.15.69:6789/0 conn(0x7fb53c0b7810 legacy=0x7fb53c0b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:35.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.500+0000 7fb540c0b640 1 -- 172.21.15.69:0/824771374 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb53c160de0 con 0x7fb53c06c5c0 2024-04-25T11:00:35.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.500+0000 7fb533fff640 1 -- 172.21.15.69:0/824771374 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 699951463 0 0) 0x7fb534001110 con 0x7fb53c0b7810 2024-04-25T11:00:35.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.501+0000 7fb540c0b640 1 --2- 172.21.15.69:0/824771374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb53c06c5c0 0x7fb53c06c9d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb52c0097a0 tx=0x7fb52c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2182828378bcd9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:35.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.501+0000 7fb533fff640 1 -- 172.21.15.69:0/824771374 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c037070 con 0x7fb53c06c5c0 2024-04-25T11:00:35.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.501+0000 7fb533fff640 1 -- 172.21.15.69:0/824771374 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb52c032440 con 0x7fb53c06c5c0 2024-04-25T11:00:35.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.501+0000 7fb533fff640 1 -- 172.21.15.69:0/824771374 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c032810 con 0x7fb53c06c5c0 2024-04-25T11:00:35.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.502+0000 7fb543670640 1 -- 172.21.15.69:0/824771374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb53c06c5c0 msgr2=0x7fb53c06c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:35.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.502+0000 7fb543670640 1 --2- 172.21.15.69:0/824771374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb53c06c5c0 0x7fb53c06c9d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb52c0097a0 tx=0x7fb52c02a180 comp rx=0 tx=0).stop 2024-04-25T11:00:35.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.502+0000 7fb543670640 1 -- 172.21.15.69:0/824771374 shutdown_connections 2024-04-25T11:00:35.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.502+0000 7fb543670640 1 --2- 172.21.15.69:0/824771374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb53c06c5c0 0x7fb53c06c9d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:35.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.502+0000 7fb543670640 1 --2- 172.21.15.69:0/824771374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb53c0d0e00 0x7fb53c0d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:35.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.502+0000 7fb543670640 1 -- 172.21.15.69:0/824771374 >> 172.21.15.69:0/824771374 conn(0x7fb53c0d5d90 msgr2=0x7fb53c0d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:35.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.502+0000 7fb543670640 1 -- 172.21.15.69:0/824771374 shutdown_connections 2024-04-25T11:00:35.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.502+0000 7fb543670640 1 -- 172.21.15.69:0/824771374 wait complete. 2024-04-25T11:00:35.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.504+0000 7fb543670640 1 Processor -- start 2024-04-25T11:00:35.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.504+0000 7fb543670640 1 -- start start 2024-04-25T11:00:35.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.505+0000 7fb543670640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb53c06c5c0 0x7fb53c1f9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:35.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.505+0000 7fb543670640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb53c0b7810 0x7fb53c0cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:35.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.505+0000 7fb543670640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb53c0d0e00 0x7fb53c0bb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:35.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.505+0000 7fb543670640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb53c160ad0 con 0x7fb53c06c5c0 2024-04-25T11:00:35.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.505+0000 7fb543670640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb53c1613a0 con 0x7fb53c0d0e00 2024-04-25T11:00:35.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.505+0000 7fb543670640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb53c0b5320 con 0x7fb53c0b7810 2024-04-25T11:00:35.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.505+0000 7fb54140c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb53c06c5c0 0x7fb53c1f9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:35.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.505+0000 7fb54140c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb53c06c5c0 0x7fb53c1f9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47750/0 (socket says 172.21.15.69:47750) 2024-04-25T11:00:35.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.505+0000 7fb54140c640 1 -- 172.21.15.69:0/2646681325 learned_addr learned my addr 172.21.15.69:0/2646681325 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:35.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.505+0000 7fb541c0d640 1 --2- 172.21.15.69:0/2646681325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb53c0d0e00 0x7fb53c0bb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:35.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.505+0000 7fb540c0b640 1 --2- 172.21.15.69:0/2646681325 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb53c0b7810 0x7fb53c0cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:35.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.505+0000 7fb54140c640 1 -- 172.21.15.69:0/2646681325 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb53c0b7810 msgr2=0x7fb53c0cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:35.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.506+0000 7fb54140c640 1 --2- 172.21.15.69:0/2646681325 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb53c0b7810 0x7fb53c0cf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:35.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.506+0000 7fb54140c640 1 -- 172.21.15.69:0/2646681325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb53c0d0e00 msgr2=0x7fb53c0bb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:35.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.506+0000 7fb54140c640 1 --2- 172.21.15.69:0/2646681325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb53c0d0e00 0x7fb53c0bb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:35.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.506+0000 7fb54140c640 1 -- 172.21.15.69:0/2646681325 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb53c157a80 con 0x7fb53c06c5c0 2024-04-25T11:00:35.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.506+0000 7fb541c0d640 1 --2- 172.21.15.69:0/2646681325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb53c0d0e00 0x7fb53c0bb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:35.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.506+0000 7fb54140c640 1 --2- 172.21.15.69:0/2646681325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb53c06c5c0 0x7fb53c1f9a00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb53800cd00 tx=0x7fb53800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:35.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.506+0000 7fb531ffb640 1 -- 172.21.15.69:0/2646681325 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb538003990 con 0x7fb53c06c5c0 2024-04-25T11:00:35.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.506+0000 7fb531ffb640 1 -- 172.21.15.69:0/2646681325 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb538003b30 con 0x7fb53c06c5c0 2024-04-25T11:00:35.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.506+0000 7fb531ffb640 1 -- 172.21.15.69:0/2646681325 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb538004020 con 0x7fb53c06c5c0 2024-04-25T11:00:35.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.506+0000 7fb543670640 1 -- 172.21.15.69:0/2646681325 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb53c0cd260 con 0x7fb53c06c5c0 2024-04-25T11:00:35.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.506+0000 7fb543670640 1 -- 172.21.15.69:0/2646681325 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb53c0cd820 con 0x7fb53c06c5c0 2024-04-25T11:00:35.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.507+0000 7fb543670640 1 -- 172.21.15.69:0/2646681325 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb53c050740 con 0x7fb53c06c5c0 2024-04-25T11:00:35.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.508+0000 7fb531ffb640 1 -- 172.21.15.69:0/2646681325 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb5380041c0 con 0x7fb53c06c5c0 2024-04-25T11:00:35.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.508+0000 7fb531ffb640 1 --2- 172.21.15.69:0/2646681325 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb5200744b0 0x7fb5200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:35.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.508+0000 7fb531ffb640 1 -- 172.21.15.69:0/2646681325 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7fb538095750 con 0x7fb53c06c5c0 2024-04-25T11:00:35.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.508+0000 7fb540c0b640 1 --2- 172.21.15.69:0/2646681325 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb5200744b0 0x7fb5200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:35.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.509+0000 7fb540c0b640 1 --2- 172.21.15.69:0/2646681325 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb5200744b0 0x7fb5200769c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb52c034000 tx=0x7fb52c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:35.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.513+0000 7fb531ffb640 1 -- 172.21.15.69:0/2646681325 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb538014030 con 0x7fb53c06c5c0 2024-04-25T11:00:35.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.723+0000 7fb543670640 1 -- 172.21.15.69:0/2646681325 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb53c0d4320 con 0x7fb53c06c5c0 2024-04-25T11:00:35.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.723+0000 7fb531ffb640 1 -- 172.21.15.69:0/2646681325 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+5180 (secure 0 0 0) 0x7fb53805fe80 con 0x7fb53c06c5c0 2024-04-25T11:00:35.724 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:00:35.724 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2223802951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":2223802951},{"type":"v1","addr":"172.21.15.146:6835","nonce":2223802951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4867,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/8243043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":8243043},{"type":"v1","addr":"172.21.15.146:6834","nonce":8243043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5250,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1639802532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1639802532},{"type":"v1","addr":"172.21.15.69:6837","nonce":1639802532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:00:06.608766+0000","modified":"2024-04-25T11:00:34.079108+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5031":{"gid":5031,"name":"c","rank":0,"incarnation":17,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6836/3766456547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3766456547},{"type":"v1","addr":"172.21.15.69:6836","nonce":3766456547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5031,"qdb_cluster":[5031]},"id":2}]} 2024-04-25T11:00:35.724 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 21 2024-04-25T11:00:35.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.725+0000 7fb543670640 1 -- 172.21.15.69:0/2646681325 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb5200744b0 msgr2=0x7fb5200769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:35.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.725+0000 7fb543670640 1 --2- 172.21.15.69:0/2646681325 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb5200744b0 0x7fb5200769c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb52c034000 tx=0x7fb52c02e000 comp rx=0 tx=0).stop 2024-04-25T11:00:35.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.725+0000 7fb543670640 1 -- 172.21.15.69:0/2646681325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb53c06c5c0 msgr2=0x7fb53c1f9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:35.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.725+0000 7fb543670640 1 --2- 172.21.15.69:0/2646681325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb53c06c5c0 0x7fb53c1f9a00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb53800cd00 tx=0x7fb53800a5c0 comp rx=0 tx=0).stop 2024-04-25T11:00:35.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.725+0000 7fb543670640 1 -- 172.21.15.69:0/2646681325 shutdown_connections 2024-04-25T11:00:35.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.725+0000 7fb543670640 1 --2- 172.21.15.69:0/2646681325 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb5200744b0 0x7fb5200769c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:35.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.725+0000 7fb543670640 1 --2- 172.21.15.69:0/2646681325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb53c0d0e00 0x7fb53c0bb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:35.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.725+0000 7fb543670640 1 --2- 172.21.15.69:0/2646681325 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb53c0b7810 0x7fb53c0cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:35.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.725+0000 7fb543670640 1 --2- 172.21.15.69:0/2646681325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb53c06c5c0 0x7fb53c1f9a00 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:35.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.725+0000 7fb543670640 1 -- 172.21.15.69:0/2646681325 >> 172.21.15.69:0/2646681325 conn(0x7fb53c0d5d90 msgr2=0x7fb53c0b7c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:35.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.726+0000 7fb543670640 1 -- 172.21.15.69:0/2646681325 shutdown_connections 2024-04-25T11:00:35.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:35.726+0000 7fb543670640 1 -- 172.21.15.69:0/2646681325 wait complete. 2024-04-25T11:00:36.070 INFO:tasks.ceph.mds.c.smithi069.stderr:2024-04-25T11:00:36.070+0000 7f8cc1b21640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:00:36.139 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:00:06.608766+0000', 'modified': '2024-04-25T11:00:34.079108+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 32, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5031}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5031': {'gid': 5031, 'name': 'c', 'rank': 0, 'incarnation': 17, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.69:6836/3766456547', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6834', 'nonce': 3766456547}, {'type': 'v1', 'addr': '172.21.15.69:6836', 'nonce': 3766456547}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5031, 'qdb_cluster': [5031]} 2024-04-25T11:00:36.139 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:00:36.139 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T11:00:36.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.248+0000 7f057c9f3640 1 Processor -- start 2024-04-25T11:00:36.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.248+0000 7f057c9f3640 1 -- start start 2024-04-25T11:00:36.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.250+0000 7f057c9f3640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f05780cfd30 0x7f05780b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.250+0000 7f057c9f3640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f05780d0140 0x7f05780b7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.250+0000 7f057c9f3640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f05780d29c0 0x7f05780d2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.250+0000 7f057c9f3640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f05780b1bc0 con 0x7f05780d29c0 2024-04-25T11:00:36.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.250+0000 7f057c9f3640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f05780cf210 con 0x7f05780cfd30 2024-04-25T11:00:36.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.250+0000 7f057c9f3640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f05780b3c40 con 0x7f05780d0140 2024-04-25T11:00:36.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.250+0000 7f0576d9d640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f05780d29c0 0x7f05780d2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:36.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.250+0000 7f0576d9d640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f05780d29c0 0x7f05780d2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47764/0 (socket says 172.21.15.69:47764) 2024-04-25T11:00:36.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.250+0000 7f0576d9d640 1 -- 172.21.15.69:0/1849569501 learned_addr learned my addr 172.21.15.69:0/1849569501 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:36.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.250+0000 7f057659c640 1 --2- 172.21.15.69:0/1849569501 >> v2:172.21.15.146:3300/0 conn(0x7f05780cfd30 0x7f05780b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:36.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.250+0000 7f0575d9b640 1 --2- 172.21.15.69:0/1849569501 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f05780d0140 0x7f05780b7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:36.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.250+0000 7f0576d9d640 1 -- 172.21.15.69:0/1849569501 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f05780d0140 msgr2=0x7f05780b7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:36.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.250+0000 7f0576d9d640 1 --2- 172.21.15.69:0/1849569501 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f05780d0140 0x7f05780b7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.250+0000 7f0576d9d640 1 -- 172.21.15.69:0/1849569501 >> v2:172.21.15.146:3300/0 conn(0x7f05780cfd30 msgr2=0x7f05780b77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:36.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.250+0000 7f0576d9d640 1 --2- 172.21.15.69:0/1849569501 >> v2:172.21.15.146:3300/0 conn(0x7f05780cfd30 0x7f05780b77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.250+0000 7f0576d9d640 1 -- 172.21.15.69:0/1849569501 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05780b3ee0 con 0x7f05780d29c0 2024-04-25T11:00:36.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.251+0000 7f0576d9d640 1 --2- 172.21.15.69:0/1849569501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05780d29c0 0x7f05780d2dd0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f05780b5530 tx=0x7f057002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e85ab8e3994de4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:36.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.251+0000 7f057559a640 1 -- 172.21.15.69:0/1849569501 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570037070 con 0x7f05780d29c0 2024-04-25T11:00:36.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.251+0000 7f057559a640 1 -- 172.21.15.69:0/1849569501 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0570032440 con 0x7f05780d29c0 2024-04-25T11:00:36.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.251+0000 7f057559a640 1 -- 172.21.15.69:0/1849569501 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570032810 con 0x7f05780d29c0 2024-04-25T11:00:36.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.252+0000 7f057c9f3640 1 -- 172.21.15.69:0/1849569501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05780d29c0 msgr2=0x7f05780d2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:36.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.252+0000 7f057c9f3640 1 --2- 172.21.15.69:0/1849569501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05780d29c0 0x7f05780d2dd0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f05780b5530 tx=0x7f057002a140 comp rx=0 tx=0).stop 2024-04-25T11:00:36.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.252+0000 7f057c9f3640 1 -- 172.21.15.69:0/1849569501 shutdown_connections 2024-04-25T11:00:36.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.252+0000 7f057c9f3640 1 --2- 172.21.15.69:0/1849569501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05780d29c0 0x7f05780d2dd0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.252+0000 7f057c9f3640 1 --2- 172.21.15.69:0/1849569501 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f05780d0140 0x7f05780b7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.252+0000 7f057c9f3640 1 --2- 172.21.15.69:0/1849569501 >> v2:172.21.15.146:3300/0 conn(0x7f05780cfd30 0x7f05780b77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.252+0000 7f057c9f3640 1 -- 172.21.15.69:0/1849569501 >> 172.21.15.69:0/1849569501 conn(0x7f057809b7d0 msgr2=0x7f057809bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:36.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.252+0000 7f057c9f3640 1 -- 172.21.15.69:0/1849569501 shutdown_connections 2024-04-25T11:00:36.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.252+0000 7f057c9f3640 1 -- 172.21.15.69:0/1849569501 wait complete. 2024-04-25T11:00:36.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.254+0000 7f057c9f3640 1 Processor -- start 2024-04-25T11:00:36.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.254+0000 7f057c9f3640 1 -- start start 2024-04-25T11:00:36.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.255+0000 7f057c9f3640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f05780cfd30 0x7f05781ea540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.255+0000 7f057c9f3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05780d0140 0x7f05780cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.255+0000 7f057c9f3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05780d29c0 0x7f05780cb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.255+0000 7f057c9f3640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f05780b20f0 con 0x7f05780d29c0 2024-04-25T11:00:36.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.255+0000 7f057c9f3640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f05780b1e10 con 0x7f05780d0140 2024-04-25T11:00:36.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.255+0000 7f057c9f3640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f05781608f0 con 0x7f05780cfd30 2024-04-25T11:00:36.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.255+0000 7f0576d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05780d29c0 0x7f05780cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:36.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.255+0000 7f0576d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05780d29c0 0x7f05780cb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47776/0 (socket says 172.21.15.69:47776) 2024-04-25T11:00:36.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.255+0000 7f0576d9d640 1 -- 172.21.15.69:0/1828094088 learned_addr learned my addr 172.21.15.69:0/1828094088 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:36.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.255+0000 7f0575d9b640 1 --2- 172.21.15.69:0/1828094088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05780d0140 0x7f05780cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:36.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.256+0000 7f057659c640 1 --2- 172.21.15.69:0/1828094088 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f05780cfd30 0x7f05781ea540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:36.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.256+0000 7f0576d9d640 1 -- 172.21.15.69:0/1828094088 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f05780cfd30 msgr2=0x7f05781ea540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:36.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.256+0000 7f0576d9d640 1 --2- 172.21.15.69:0/1828094088 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f05780cfd30 0x7f05781ea540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.256+0000 7f0576d9d640 1 -- 172.21.15.69:0/1828094088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05780d0140 msgr2=0x7f05780cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:36.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.256+0000 7f0576d9d640 1 --2- 172.21.15.69:0/1828094088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05780d0140 0x7f05780cab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.256+0000 7f0576d9d640 1 -- 172.21.15.69:0/1828094088 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05781eaf20 con 0x7f05780d29c0 2024-04-25T11:00:36.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.256+0000 7f0576d9d640 1 --2- 172.21.15.69:0/1828094088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05780d29c0 0x7f05780cb070 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f057002a180 tx=0x7f05700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:36.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.256+0000 7f0566ffd640 1 -- 172.21.15.69:0/1828094088 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570037040 con 0x7f05780d29c0 2024-04-25T11:00:36.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.256+0000 7f0566ffd640 1 -- 172.21.15.69:0/1828094088 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0570042070 con 0x7f05780d29c0 2024-04-25T11:00:36.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.256+0000 7f057c9f3640 1 -- 172.21.15.69:0/1828094088 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05781eb200 con 0x7f05780d29c0 2024-04-25T11:00:36.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.256+0000 7f0566ffd640 1 -- 172.21.15.69:0/1828094088 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570030920 con 0x7f05780d29c0 2024-04-25T11:00:36.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.256+0000 7f057c9f3640 1 -- 172.21.15.69:0/1828094088 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05781eb7c0 con 0x7f05780d29c0 2024-04-25T11:00:36.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.258+0000 7f0566ffd640 1 -- 172.21.15.69:0/1828094088 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0570030ac0 con 0x7f05780d29c0 2024-04-25T11:00:36.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.258+0000 7f0566ffd640 1 --2- 172.21.15.69:0/1828094088 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f05580744b0 0x7f05580769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.258+0000 7f057659c640 1 --2- 172.21.15.69:0/1828094088 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f05580744b0 0x7f05580769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:36.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.258+0000 7f0566ffd640 1 -- 172.21.15.69:0/1828094088 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7f05700b5b70 con 0x7f05780d29c0 2024-04-25T11:00:36.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.259+0000 7f057659c640 1 --2- 172.21.15.69:0/1828094088 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f05580744b0 0x7f05580769c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f05780ccd40 tx=0x7f05680097b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:36.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.259+0000 7f058397e740 1 Processor -- start 2024-04-25T11:00:36.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.260+0000 7f058397e740 1 -- start start 2024-04-25T11:00:36.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.261+0000 7f058397e740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55f7dd4b0270 0x55f7dd87c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.261+0000 7f058397e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7dd4873b0 0x55f7dd878fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.261+0000 7f058397e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f7dd48f590 0x55f7dd852260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.261+0000 7f058397e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f7dd7b6740 con 0x55f7dd4873b0 2024-04-25T11:00:36.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.261+0000 7f058397e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55f7dd7523a0 con 0x55f7dd48f590 2024-04-25T11:00:36.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.261+0000 7f058397e740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x55f7dd7b2700 con 0x55f7dd4b0270 2024-04-25T11:00:36.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.261+0000 7f0575d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7dd4873b0 0x55f7dd878fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:36.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.261+0000 7f0575d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7dd4873b0 0x55f7dd878fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47778/0 (socket says 172.21.15.69:47778) 2024-04-25T11:00:36.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.261+0000 7f0575d9b640 1 -- 172.21.15.69:0/3262618830 learned_addr learned my addr 172.21.15.69:0/3262618830 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:36.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.261+0000 7f057659c640 1 --2- 172.21.15.69:0/3262618830 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55f7dd4b0270 0x55f7dd87c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:36.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.261+0000 7f0575d9b640 1 -- 172.21.15.69:0/3262618830 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55f7dd4b0270 msgr2=0x55f7dd87c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:36.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.261+0000 7f0576d9d640 1 --2- 172.21.15.69:0/3262618830 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f7dd48f590 0x55f7dd852260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:36.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.261+0000 7f0575d9b640 1 --2- 172.21.15.69:0/3262618830 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55f7dd4b0270 0x55f7dd87c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.261+0000 7f0575d9b640 1 -- 172.21.15.69:0/3262618830 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f7dd48f590 msgr2=0x55f7dd852260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:36.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.261+0000 7f0575d9b640 1 --2- 172.21.15.69:0/3262618830 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f7dd48f590 0x55f7dd852260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.261+0000 7f0575d9b640 1 -- 172.21.15.69:0/3262618830 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7dd76e750 con 0x55f7dd4873b0 2024-04-25T11:00:36.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.261+0000 7f0575d9b640 1 --2- 172.21.15.69:0/3262618830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7dd4873b0 0x55f7dd878fa0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f056c0078f0 tx=0x7f056c007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd0b0fb74e79f39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:36.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.261+0000 7f0557fff640 1 -- 172.21.15.69:0/3262618830 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c018070 con 0x55f7dd4873b0 2024-04-25T11:00:36.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.262+0000 7f0557fff640 1 -- 172.21.15.69:0/3262618830 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f056c0109e0 con 0x55f7dd4873b0 2024-04-25T11:00:36.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.262+0000 7f0557fff640 1 -- 172.21.15.69:0/3262618830 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c010db0 con 0x55f7dd4873b0 2024-04-25T11:00:36.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.262+0000 7f058397e740 1 -- 172.21.15.69:0/3262618830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7dd4873b0 msgr2=0x55f7dd878fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:36.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.262+0000 7f058397e740 1 --2- 172.21.15.69:0/3262618830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7dd4873b0 0x55f7dd878fa0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f056c0078f0 tx=0x7f056c007db0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.262+0000 7f058397e740 1 -- 172.21.15.69:0/3262618830 shutdown_connections 2024-04-25T11:00:36.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.262+0000 7f058397e740 1 --2- 172.21.15.69:0/3262618830 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f7dd48f590 0x55f7dd852260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.262+0000 7f058397e740 1 --2- 172.21.15.69:0/3262618830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7dd4873b0 0x55f7dd878fa0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.262+0000 7f058397e740 1 --2- 172.21.15.69:0/3262618830 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55f7dd4b0270 0x55f7dd87c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.262+0000 7f058397e740 1 -- 172.21.15.69:0/3262618830 >> 172.21.15.69:0/3262618830 conn(0x55f7dd4a43c0 msgr2=0x55f7dd8519f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:36.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.262+0000 7f058397e740 1 -- 172.21.15.69:0/3262618830 shutdown_connections 2024-04-25T11:00:36.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.262+0000 7f058397e740 1 -- 172.21.15.69:0/3262618830 wait complete. 2024-04-25T11:00:36.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.263+0000 7f058397e740 1 -- start start 2024-04-25T11:00:36.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.263+0000 7f058397e740 1 Processor -- start 2024-04-25T11:00:36.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.265+0000 7f058397e740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55f7dd4873b0 0x55f7dd895ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.265+0000 7f058397e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f7dd48f590 0x55f7dd896410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.265+0000 7f058397e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7dd4b0270 0x55f7dd87d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.265+0000 7f058397e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f7dd73dfe0 con 0x55f7dd4b0270 2024-04-25T11:00:36.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.265+0000 7f058397e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55f7dd8a6a90 con 0x55f7dd48f590 2024-04-25T11:00:36.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.265+0000 7f058397e740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x55f7dd8969e0 con 0x55f7dd4873b0 2024-04-25T11:00:36.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.265+0000 7f058397e740 10 client.0 _ll_register_callbacks cb 0x55f7dd754150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:00:36.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.265+0000 7f0576d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7dd4b0270 0x55f7dd87d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:36.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.265+0000 7f0576d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7dd4b0270 0x55f7dd87d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47780/0 (socket says 172.21.15.69:47780) 2024-04-25T11:00:36.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.265+0000 7f0576d9d640 1 -- 172.21.15.69:0/4144801928 learned_addr learned my addr 172.21.15.69:0/4144801928 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:36.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.265+0000 7f0575d9b640 1 --2- 172.21.15.69:0/4144801928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f7dd48f590 0x55f7dd896410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:36.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.265+0000 7f057659c640 1 --2- 172.21.15.69:0/4144801928 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55f7dd4873b0 0x55f7dd895ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:36.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.265+0000 7f0576d9d640 1 -- 172.21.15.69:0/4144801928 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55f7dd4873b0 msgr2=0x55f7dd895ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:36.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.265+0000 7f0576d9d640 1 --2- 172.21.15.69:0/4144801928 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55f7dd4873b0 0x55f7dd895ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.265+0000 7f0576d9d640 1 -- 172.21.15.69:0/4144801928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f7dd48f590 msgr2=0x55f7dd896410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:36.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.265+0000 7f0576d9d640 1 --2- 172.21.15.69:0/4144801928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f7dd48f590 0x55f7dd896410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.265+0000 7f0576d9d640 1 -- 172.21.15.69:0/4144801928 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7dd4c82d0 con 0x55f7dd4b0270 2024-04-25T11:00:36.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.266+0000 7f0576d9d640 1 --2- 172.21.15.69:0/4144801928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7dd4b0270 0x55f7dd87d2a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f05700b3fe0 tx=0x7f057007fb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d1b453b3e83503 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:36.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.266+0000 7f05557fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-25T11:00:36.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.266+0000 7f05557fa640 1 -- 172.21.15.69:0/4144801928 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570088340 con 0x55f7dd4b0270 2024-04-25T11:00:36.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.266+0000 7f05557fa640 1 -- 172.21.15.69:0/4144801928 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7f057007eeb0 con 0x55f7dd4b0270 2024-04-25T11:00:36.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.266+0000 7f058397e740 1 -- 172.21.15.69:0/4144801928 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f056c0109e0 con 0x55f7dd4b0270 2024-04-25T11:00:36.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.266+0000 7f05557fa640 1 -- 172.21.15.69:0/4144801928 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f057007f760 con 0x55f7dd4b0270 2024-04-25T11:00:36.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.266+0000 7f058397e740 10 client.5289 fetch_fsmap learned FSMap version 21 2024-04-25T11:00:36.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.266+0000 7f058397e740 1 -- 172.21.15.69:0/4144801928 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55f7dd5926b0 con 0x55f7dd4b0270 2024-04-25T11:00:36.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.266+0000 7f05557fa640 1 -- 172.21.15.69:0/4144801928 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f056c0109e0 con 0x55f7dd4b0270 2024-04-25T11:00:36.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.266+0000 7f05557fa640 1 -- 172.21.15.69:0/4144801928 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f057003a070 con 0x55f7dd4b0270 2024-04-25T11:00:36.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.266+0000 7f058397e740 10 client.5289 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T11:00:36.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.266+0000 7f058397e740 10 client.5289 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:00:36.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.266+0000 7f058397e740 20 client.5289 populate_metadata read hostname 'smithi069' 2024-04-25T11:00:36.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.266+0000 7f058397e740 1 --2- 172.21.15.69:0/4144801928 >> [v2:172.21.15.69:6834/3766456547,v1:172.21.15.69:6836/3766456547] conn(0x55f7dd46ee40 0x55f7dd87f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.266+0000 7f058397e740 4 client.5289 mds_command: new command op to 5031 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:00:36.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.266+0000 7f058397e740 1 -- 172.21.15.69:0/4144801928 --> [v2:172.21.15.69:6834/3766456547,v1:172.21.15.69:6836/3766456547] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7dd7c4350 con 0x55f7dd46ee40 2024-04-25T11:00:36.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.266+0000 7f057659c640 1 --2- 172.21.15.69:0/4144801928 >> [v2:172.21.15.69:6834/3766456547,v1:172.21.15.69:6836/3766456547] conn(0x55f7dd46ee40 0x55f7dd87f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:36.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.267+0000 7f05557fa640 10 client.5289 ms_handle_connect on v2:172.21.15.69:6834/3766456547 2024-04-25T11:00:36.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.267+0000 7f057659c640 1 --2- 172.21.15.69:0/4144801928 >> [v2:172.21.15.69:6834/3766456547,v1:172.21.15.69:6836/3766456547] conn(0x55f7dd46ee40 0x55f7dd87f500 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10f19bac088056c0 server_cookie=becf9376607914ab in_seq=0 out_seq=0 2024-04-25T11:00:36.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.267+0000 7f05557fa640 1 -- 172.21.15.69:0/4144801928 <== mds.0 v2:172.21.15.69:6834/3766456547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f7dd7c4350 con 0x55f7dd46ee40 2024-04-25T11:00:36.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.267+0000 7f05557fa640 10 client.5289 handle_command_reply: tid=0 2024-04-25T11:00:36.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.267+0000 7f058397e740 1 client.5289 shutdown 2024-04-25T11:00:36.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.268+0000 7f058397e740 1 -- 172.21.15.69:0/4144801928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7dd4b0270 msgr2=0x55f7dd87d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:36.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.268+0000 7f058397e740 1 --2- 172.21.15.69:0/4144801928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7dd4b0270 0x55f7dd87d2a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f05700b3fe0 tx=0x7f057007fb10 comp rx=0 tx=0).stop 2024-04-25T11:00:36.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.268+0000 7f058397e740 1 -- 172.21.15.69:0/4144801928 shutdown_connections 2024-04-25T11:00:36.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.268+0000 7f058397e740 1 --2- 172.21.15.69:0/4144801928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7dd4b0270 0x55f7dd87d2a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.268+0000 7f058397e740 1 --2- 172.21.15.69:0/4144801928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f7dd48f590 0x55f7dd896410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.268+0000 7f058397e740 1 --2- 172.21.15.69:0/4144801928 >> [v2:172.21.15.69:6834/3766456547,v1:172.21.15.69:6836/3766456547] conn(0x55f7dd46ee40 0x55f7dd87f500 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.268+0000 7f058397e740 1 --2- 172.21.15.69:0/4144801928 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55f7dd4873b0 0x55f7dd895ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.268+0000 7f05557fa640 0 client.5289 ms_handle_reset on v2:172.21.15.69:6834/3766456547 2024-04-25T11:00:36.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.268+0000 7f058397e740 1 -- 172.21.15.69:0/4144801928 >> 172.21.15.69:0/4144801928 conn(0x55f7dd4a43c0 msgr2=0x55f7dd850d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:36.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.268+0000 7f058397e740 1 -- 172.21.15.69:0/4144801928 shutdown_connections 2024-04-25T11:00:36.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.268+0000 7f058397e740 1 -- 172.21.15.69:0/4144801928 wait complete. 2024-04-25T11:00:36.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.268+0000 7f058397e740 20 client.5289 trim_cache size 0 max 16384 2024-04-25T11:00:36.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.286+0000 7f058397e740 1 Processor -- start 2024-04-25T11:00:36.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.286+0000 7f058397e740 1 -- start start 2024-04-25T11:00:36.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.288+0000 7f058397e740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55f7dd4873b0 0x55f7dd87bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.288+0000 7f058397e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f7dd48f590 0x55f7dd8ba250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.288+0000 7f058397e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7dd4b0270 0x55f7dd853110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.288+0000 7f058397e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0578201ea0 con 0x55f7dd4b0270 2024-04-25T11:00:36.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.288+0000 7f058397e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0578201200 con 0x55f7dd48f590 2024-04-25T11:00:36.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.288+0000 7f058397e740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f0578201850 con 0x55f7dd4873b0 2024-04-25T11:00:36.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.288+0000 7f0576d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7dd4b0270 0x55f7dd853110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:36.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.288+0000 7f0576d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7dd4b0270 0x55f7dd853110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47782/0 (socket says 172.21.15.69:47782) 2024-04-25T11:00:36.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.288+0000 7f0576d9d640 1 -- 172.21.15.69:0/4095154851 learned_addr learned my addr 172.21.15.69:0/4095154851 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:36.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.288+0000 7f0576d9d640 1 -- 172.21.15.69:0/4095154851 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55f7dd4873b0 msgr2=0x55f7dd87bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:36.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.288+0000 7f0576d9d640 1 --2- 172.21.15.69:0/4095154851 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55f7dd4873b0 0x55f7dd87bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.288+0000 7f0576d9d640 1 -- 172.21.15.69:0/4095154851 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f7dd48f590 msgr2=0x55f7dd8ba250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:36.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.288+0000 7f0576d9d640 1 --2- 172.21.15.69:0/4095154851 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f7dd48f590 0x55f7dd8ba250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.288+0000 7f0576d9d640 1 -- 172.21.15.69:0/4095154851 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7dd879cc0 con 0x55f7dd4b0270 2024-04-25T11:00:36.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.288+0000 7f0576d9d640 1 --2- 172.21.15.69:0/4095154851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7dd4b0270 0x55f7dd853110 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f057007bcd0 tx=0x7f0570081000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a93abd35f9d949c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:36.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.288+0000 7f05577fe640 1 -- 172.21.15.69:0/4095154851 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057009e020 con 0x55f7dd4b0270 2024-04-25T11:00:36.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.288+0000 7f05577fe640 1 -- 172.21.15.69:0/4095154851 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0570083f50 con 0x55f7dd4b0270 2024-04-25T11:00:36.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.288+0000 7f05577fe640 1 -- 172.21.15.69:0/4095154851 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570088340 con 0x55f7dd4b0270 2024-04-25T11:00:36.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.289+0000 7f058397e740 1 -- 172.21.15.69:0/4095154851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7dd4b0270 msgr2=0x55f7dd853110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:36.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.289+0000 7f058397e740 1 --2- 172.21.15.69:0/4095154851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7dd4b0270 0x55f7dd853110 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f057007bcd0 tx=0x7f0570081000 comp rx=0 tx=0).stop 2024-04-25T11:00:36.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.289+0000 7f058397e740 1 -- 172.21.15.69:0/4095154851 shutdown_connections 2024-04-25T11:00:36.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.289+0000 7f058397e740 1 --2- 172.21.15.69:0/4095154851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7dd4b0270 0x55f7dd853110 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.289+0000 7f058397e740 1 --2- 172.21.15.69:0/4095154851 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f7dd48f590 0x55f7dd8ba250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.289+0000 7f058397e740 1 --2- 172.21.15.69:0/4095154851 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55f7dd4873b0 0x55f7dd87bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.289+0000 7f058397e740 1 -- 172.21.15.69:0/4095154851 >> 172.21.15.69:0/4095154851 conn(0x55f7dd4a43c0 msgr2=0x55f7dd87e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:36.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.289+0000 7f058397e740 1 -- 172.21.15.69:0/4095154851 shutdown_connections 2024-04-25T11:00:36.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.289+0000 7f058397e740 1 -- 172.21.15.69:0/4095154851 wait complete. 2024-04-25T11:00:36.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.290+0000 7f058397e740 1 -- start start 2024-04-25T11:00:36.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.290+0000 7f058397e740 1 Processor -- start 2024-04-25T11:00:36.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.291+0000 7f058397e740 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55f7dd4873b0 0x55f7dd87f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.291+0000 7f058397e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7dd48f590 0x55f7dd87f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.291+0000 7f058397e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f7dd4b0270 0x55f7dd8954d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.291+0000 7f058397e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f7dd878620 con 0x55f7dd48f590 2024-04-25T11:00:36.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.291+0000 7f058397e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55f7dd8a67d0 con 0x55f7dd4b0270 2024-04-25T11:00:36.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.291+0000 7f058397e740 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x55f7dd879230 con 0x55f7dd4873b0 2024-04-25T11:00:36.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.292+0000 7f058397e740 10 client.0 _ll_register_callbacks cb 0x7f05700092b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:00:36.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.292+0000 7f0575d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7dd48f590 0x55f7dd87f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:36.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.292+0000 7f0575d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7dd48f590 0x55f7dd87f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47794/0 (socket says 172.21.15.69:47794) 2024-04-25T11:00:36.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.292+0000 7f0575d9b640 1 -- 172.21.15.69:0/776513599 learned_addr learned my addr 172.21.15.69:0/776513599 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:36.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.292+0000 7f057659c640 1 --2- 172.21.15.69:0/776513599 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55f7dd4873b0 0x55f7dd87f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:36.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.292+0000 7f0576d9d640 1 --2- 172.21.15.69:0/776513599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f7dd4b0270 0x55f7dd8954d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:36.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.292+0000 7f0575d9b640 1 -- 172.21.15.69:0/776513599 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55f7dd4873b0 msgr2=0x55f7dd87f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:36.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.292+0000 7f0575d9b640 1 --2- 172.21.15.69:0/776513599 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55f7dd4873b0 0x55f7dd87f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.292+0000 7f0575d9b640 1 -- 172.21.15.69:0/776513599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f7dd4b0270 msgr2=0x55f7dd8954d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:36.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.292+0000 7f0575d9b640 1 --2- 172.21.15.69:0/776513599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f7dd4b0270 0x55f7dd8954d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.292+0000 7f0575d9b640 1 -- 172.21.15.69:0/776513599 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7dd87c070 con 0x55f7dd48f590 2024-04-25T11:00:36.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.292+0000 7f0576d9d640 1 --2- 172.21.15.69:0/776513599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f7dd4b0270 0x55f7dd8954d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:00:36.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.292+0000 7f0575d9b640 1 --2- 172.21.15.69:0/776513599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7dd48f590 0x55f7dd87f9c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f056c017000 tx=0x7f056c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8240be6ec82be897 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:36.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.292+0000 7f05557fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-25T11:00:36.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.292+0000 7f05557fa640 1 -- 172.21.15.69:0/776513599 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c018040 con 0x55f7dd48f590 2024-04-25T11:00:36.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.292+0000 7f05557fa640 1 -- 172.21.15.69:0/776513599 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7f056c00f040 con 0x55f7dd48f590 2024-04-25T11:00:36.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.292+0000 7f058397e740 1 -- 172.21.15.69:0/776513599 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0570083f50 con 0x55f7dd48f590 2024-04-25T11:00:36.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.292+0000 7f05557fa640 1 -- 172.21.15.69:0/776513599 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f056c02f050 con 0x55f7dd48f590 2024-04-25T11:00:36.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.292+0000 7f058397e740 10 client.5295 fetch_fsmap learned FSMap version 21 2024-04-25T11:00:36.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.292+0000 7f058397e740 1 -- 172.21.15.69:0/776513599 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55f7dd914e60 con 0x55f7dd48f590 2024-04-25T11:00:36.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.293+0000 7f05557fa640 1 -- 172.21.15.69:0/776513599 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0570083f50 con 0x55f7dd48f590 2024-04-25T11:00:36.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.293+0000 7f05557fa640 1 -- 172.21.15.69:0/776513599 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f056c032020 con 0x55f7dd48f590 2024-04-25T11:00:36.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.293+0000 7f058397e740 10 client.5295 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T11:00:36.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.293+0000 7f058397e740 10 client.5295 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:00:36.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.293+0000 7f058397e740 20 client.5295 populate_metadata read hostname 'smithi069' 2024-04-25T11:00:36.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.293+0000 7f058397e740 1 --2- 172.21.15.69:0/776513599 >> [v2:172.21.15.69:6834/3766456547,v1:172.21.15.69:6836/3766456547] conn(0x55f7dd853480 0x55f7dd913960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.293+0000 7f058397e740 4 client.5295 mds_command: new command op to 5031 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:00:36.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.293+0000 7f058397e740 1 -- 172.21.15.69:0/776513599 --> [v2:172.21.15.69:6834/3766456547,v1:172.21.15.69:6836/3766456547] -- command(tid 0: {"prefix": "status"}) -- 0x55f7dd7be1c0 con 0x55f7dd853480 2024-04-25T11:00:36.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.293+0000 7f057659c640 1 --2- 172.21.15.69:0/776513599 >> [v2:172.21.15.69:6834/3766456547,v1:172.21.15.69:6836/3766456547] conn(0x55f7dd853480 0x55f7dd913960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:36.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.293+0000 7f05557fa640 10 client.5295 ms_handle_connect on v2:172.21.15.69:6834/3766456547 2024-04-25T11:00:36.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.293+0000 7f057659c640 1 --2- 172.21.15.69:0/776513599 >> [v2:172.21.15.69:6834/3766456547,v1:172.21.15.69:6836/3766456547] conn(0x55f7dd853480 0x55f7dd913960 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b681e042374fa31f server_cookie=e1c94ddbc70b8daf in_seq=0 out_seq=0 2024-04-25T11:00:36.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.293+0000 7f05557fa640 1 -- 172.21.15.69:0/776513599 <== mds.0 v2:172.21.15.69:6834/3766456547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55f7dd7be1c0 con 0x55f7dd853480 2024-04-25T11:00:36.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.293+0000 7f05557fa640 10 client.5295 handle_command_reply: tid=0 2024-04-25T11:00:36.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.293+0000 7f058397e740 1 client.5295 shutdown 2024-04-25T11:00:36.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.294+0000 7f058397e740 1 -- 172.21.15.69:0/776513599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7dd48f590 msgr2=0x55f7dd87f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:36.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.294+0000 7f058397e740 1 --2- 172.21.15.69:0/776513599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7dd48f590 0x55f7dd87f9c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f056c017000 tx=0x7f056c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.294+0000 7f058397e740 1 -- 172.21.15.69:0/776513599 shutdown_connections 2024-04-25T11:00:36.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.294+0000 7f058397e740 1 --2- 172.21.15.69:0/776513599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f7dd4b0270 0x55f7dd8954d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.294+0000 7f058397e740 1 --2- 172.21.15.69:0/776513599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f7dd48f590 0x55f7dd87f9c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.294+0000 7f058397e740 1 --2- 172.21.15.69:0/776513599 >> [v2:172.21.15.69:6834/3766456547,v1:172.21.15.69:6836/3766456547] conn(0x55f7dd853480 0x55f7dd913960 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.294+0000 7f058397e740 1 --2- 172.21.15.69:0/776513599 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55f7dd4873b0 0x55f7dd87f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.294+0000 7f05557fa640 0 client.5295 ms_handle_reset on v2:172.21.15.69:6834/3766456547 2024-04-25T11:00:36.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.294+0000 7f058397e740 1 -- 172.21.15.69:0/776513599 >> 172.21.15.69:0/776513599 conn(0x55f7dd4a43c0 msgr2=0x55f7dd87c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:36.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.294+0000 7f058397e740 1 -- 172.21.15.69:0/776513599 shutdown_connections 2024-04-25T11:00:36.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.294+0000 7f058397e740 1 -- 172.21.15.69:0/776513599 wait complete. 2024-04-25T11:00:36.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.294+0000 7f058397e740 20 client.5295 trim_cache size 0 max 16384 2024-04-25T11:00:36.300 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T11:00:36.300 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "ed0c8fe4-1bbb-4705-be72-eeea5bef859e", 2024-04-25T11:00:36.300 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 0, 2024-04-25T11:00:36.300 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 5031, 2024-04-25T11:00:36.300 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-04-25T11:00:36.300 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-04-25T11:00:36.300 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-04-25T11:00:36.301 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 6.220105674, 2024-04-25T11:00:36.301 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 21, 2024-04-25T11:00:36.301 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 32, 2024-04-25T11:00:36.301 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 32, 2024-04-25T11:00:36.301 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 36.078545378000001 2024-04-25T11:00:36.301 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T11:00:36.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.294+0000 7f057c9f3640 1 -- 172.21.15.69:0/1828094088 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f05580744b0 msgr2=0x7f05580769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:36.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.294+0000 7f057c9f3640 1 --2- 172.21.15.69:0/1828094088 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f05580744b0 0x7f05580769c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f05780ccd40 tx=0x7f05680097b0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.294+0000 7f057c9f3640 1 -- 172.21.15.69:0/1828094088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05780d29c0 msgr2=0x7f05780cb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:36.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.294+0000 7f057c9f3640 1 --2- 172.21.15.69:0/1828094088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05780d29c0 0x7f05780cb070 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f057002a180 tx=0x7f05700046c0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.294+0000 7f057c9f3640 1 -- 172.21.15.69:0/1828094088 shutdown_connections 2024-04-25T11:00:36.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.294+0000 7f057c9f3640 1 --2- 172.21.15.69:0/1828094088 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f05580744b0 0x7f05580769c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.294+0000 7f057c9f3640 1 --2- 172.21.15.69:0/1828094088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05780d29c0 0x7f05780cb070 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.294+0000 7f057c9f3640 1 --2- 172.21.15.69:0/1828094088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05780d0140 0x7f05780cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.294+0000 7f057c9f3640 1 --2- 172.21.15.69:0/1828094088 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f05780cfd30 0x7f05781ea540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.294+0000 7f057c9f3640 1 -- 172.21.15.69:0/1828094088 >> 172.21.15.69:0/1828094088 conn(0x7f057809b7d0 msgr2=0x7f0578070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:36.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.294+0000 7f057c9f3640 1 -- 172.21.15.69:0/1828094088 shutdown_connections 2024-04-25T11:00:36.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.294+0000 7f057c9f3640 1 -- 172.21.15.69:0/1828094088 wait complete. 2024-04-25T11:00:36.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:00:36.306 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%a /home/ubuntu/cephtest/mnt.0/testdir/testfile) 2024-04-25T11:00:36.322 INFO:teuthology.orchestra.run.smithi069.stdout:600 2024-04-25T11:00:36.322 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:00:36.322 DEBUG:teuthology.orchestra.run.smithi069:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:00:36.401 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:00:36.443 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:00:36.443 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:00:36.443 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T11:00:36.443 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:00:36.458 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T11:00:36.459 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:00:36.514 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:00:36.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.659+0000 7f36b3fb2640 1 Processor -- start 2024-04-25T11:00:36.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.660+0000 7f36b3fb2640 1 -- start start 2024-04-25T11:00:36.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.661+0000 7f36b3fb2640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f36ac0b6a00 0x7f36ac15d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.661+0000 7f36b3fb2640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f36ac06d560 0x7f36ac0b4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.661+0000 7f36b3fb2640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36ac1611e0 con 0x7f36ac15a2f0 2024-04-25T11:00:36.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.661+0000 7f36b3fb2640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f36ac160d50 con 0x7f36ac06d560 2024-04-25T11:00:36.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.661+0000 7f36b3fb2640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f36ac15a700 con 0x7f36ac0b6a00 2024-04-25T11:00:36.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.661+0000 7f36b154d640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f36ac15a2f0 0x7f36ac06ce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:39414/0 (socket says 172.21.15.69:39414) 2024-04-25T11:00:36.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.661+0000 7f36b154d640 1 -- 172.21.15.69:0/3570010443 learned_addr learned my addr 172.21.15.69:0/3570010443 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:36.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.661+0000 7f36b1d4e640 1 --2- 172.21.15.69:0/3570010443 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f36ac0b6a00 0x7f36ac15d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:36.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.661+0000 7f36b254f640 1 --2- 172.21.15.69:0/3570010443 >> v2:172.21.15.146:3300/0 conn(0x7f36ac06d560 0x7f36ac0b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:36.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.661+0000 7f36b0d4c640 1 -- 172.21.15.69:0/3570010443 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3577179151 0 0) 0x7f36ac1611e0 con 0x7f36ac15a2f0 2024-04-25T11:00:36.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.661+0000 7f36b0d4c640 1 -- 172.21.15.69:0/3570010443 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3698004630 con 0x7f36ac15a2f0 2024-04-25T11:00:36.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.662+0000 7f36b0d4c640 1 -- 172.21.15.69:0/3570010443 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 409071031 0 0) 0x7f3698004630 con 0x7f36ac15a2f0 2024-04-25T11:00:36.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.662+0000 7f36b0d4c640 1 -- 172.21.15.69:0/3570010443 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f36ac0b6a00 msgr2=0x7f36ac15d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:36.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.662+0000 7f36b0d4c640 1 --2- 172.21.15.69:0/3570010443 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f36ac0b6a00 0x7f36ac15d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.662+0000 7f36b0d4c640 1 -- 172.21.15.69:0/3570010443 >> v2:172.21.15.146:3300/0 conn(0x7f36ac06d560 msgr2=0x7f36ac0b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:36.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.662+0000 7f36b0d4c640 1 --2- 172.21.15.69:0/3570010443 >> v2:172.21.15.146:3300/0 conn(0x7f36ac06d560 0x7f36ac0b4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.662+0000 7f36b0d4c640 1 -- 172.21.15.69:0/3570010443 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36ac160420 con 0x7f36ac15a2f0 2024-04-25T11:00:36.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.662+0000 7f36b0d4c640 1 -- 172.21.15.69:0/3570010443 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7f369c001670 con 0x7f36ac15a2f0 2024-04-25T11:00:36.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.662+0000 7f36b0d4c640 1 -- 172.21.15.69:0/3570010443 >> v1:172.21.15.69:6789/0 conn(0x7f36ac15a2f0 legacy=0x7f36ac06ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:36.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.662+0000 7f36b0d4c640 1 --2- 172.21.15.69:0/3570010443 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3698004ac0 0x7f3698004f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.662+0000 7f36b0d4c640 1 --2- 172.21.15.69:0/3570010443 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3698005910 0x7f3698005d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.662+0000 7f36b0d4c640 1 --2- 172.21.15.69:0/3570010443 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f36ac0b6a00 0x7f3698006260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.662+0000 7f36b0d4c640 1 -- 172.21.15.69:0/3570010443 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f36ac160d50 con 0x7f3698005910 2024-04-25T11:00:36.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.662+0000 7f36b0d4c640 1 -- 172.21.15.69:0/3570010443 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f36ac15a700 con 0x7f3698004ac0 2024-04-25T11:00:36.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.662+0000 7f36b0d4c640 1 -- 172.21.15.69:0/3570010443 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f369800cad0 con 0x7f36ac0b6a00 2024-04-25T11:00:36.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.662+0000 7f36b154d640 1 --2- 172.21.15.69:0/3570010443 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3698005910 0x7f3698005d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:36.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.662+0000 7f36b1d4e640 1 --2- 172.21.15.69:0/3570010443 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3698004ac0 0x7f3698004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:36.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.662+0000 7f36b254f640 1 --2- 172.21.15.69:0/3570010443 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f36ac0b6a00 0x7f3698006260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:36.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.662+0000 7f36b154d640 1 -- 172.21.15.69:0/3570010443 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f36ac0b6a00 msgr2=0x7f3698006260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:36.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.662+0000 7f36b154d640 1 --2- 172.21.15.69:0/3570010443 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f36ac0b6a00 0x7f3698006260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.662+0000 7f36b154d640 1 -- 172.21.15.69:0/3570010443 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3698004ac0 msgr2=0x7f3698004f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:36.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.662+0000 7f36b154d640 1 --2- 172.21.15.69:0/3570010443 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3698004ac0 0x7f3698004f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.662+0000 7f36b154d640 1 -- 172.21.15.69:0/3570010443 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3698004630 con 0x7f3698005910 2024-04-25T11:00:36.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.662+0000 7f36b254f640 1 --2- 172.21.15.69:0/3570010443 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f36ac0b6a00 0x7f3698006260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:36.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.663+0000 7f36b154d640 1 --2- 172.21.15.69:0/3570010443 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3698005910 0x7f3698005d20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f369c007fd0 tx=0x7f3698006940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=487522c8e2acca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:36.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.663+0000 7f36b0d4c640 1 -- 172.21.15.69:0/3570010443 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369c007d50 con 0x7f3698005910 2024-04-25T11:00:36.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.663+0000 7f36b0d4c640 1 -- 172.21.15.69:0/3570010443 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f369c005ad0 con 0x7f3698005910 2024-04-25T11:00:36.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.663+0000 7f36b0d4c640 1 -- 172.21.15.69:0/3570010443 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369c007070 con 0x7f3698005910 2024-04-25T11:00:36.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.664+0000 7f36b3fb2640 1 -- 172.21.15.69:0/3570010443 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3698005910 msgr2=0x7f3698005d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:36.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.664+0000 7f36b3fb2640 1 --2- 172.21.15.69:0/3570010443 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3698005910 0x7f3698005d20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f369c007fd0 tx=0x7f3698006940 comp rx=0 tx=0).stop 2024-04-25T11:00:36.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.664+0000 7f36b254f640 1 -- 172.21.15.69:0/3570010443 reap_dead start 2024-04-25T11:00:36.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.664+0000 7f36b3fb2640 1 -- 172.21.15.69:0/3570010443 shutdown_connections 2024-04-25T11:00:36.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.664+0000 7f36b3fb2640 1 -- 172.21.15.69:0/3570010443 >> 172.21.15.69:0/3570010443 conn(0x7f36ac0d1ae0 msgr2=0x7f36ac0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:36.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.664+0000 7f36b3fb2640 1 -- 172.21.15.69:0/3570010443 shutdown_connections 2024-04-25T11:00:36.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.664+0000 7f36b3fb2640 1 -- 172.21.15.69:0/3570010443 wait complete. 2024-04-25T11:00:36.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.666+0000 7f36b3fb2640 1 Processor -- start 2024-04-25T11:00:36.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.666+0000 7f36b3fb2640 1 -- start start 2024-04-25T11:00:36.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.667+0000 7f36b3fb2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3698005910 0x7f36ac0b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.667+0000 7f36b3fb2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36ac15f150 0x7f36ac0b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.667+0000 7f36b3fb2640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f36ac157070 0x7f36ac0ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.667+0000 7f36b3fb2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f36ac0ce580 con 0x7f36ac15f150 2024-04-25T11:00:36.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.667+0000 7f36b3fb2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f36ac1609e0 con 0x7f3698005910 2024-04-25T11:00:36.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.667+0000 7f36b3fb2640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f36ac0ce700 con 0x7f36ac157070 2024-04-25T11:00:36.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.667+0000 7f36b154d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36ac15f150 0x7f36ac0b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:36.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.667+0000 7f36b154d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36ac15f150 0x7f36ac0b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47808/0 (socket says 172.21.15.69:47808) 2024-04-25T11:00:36.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.667+0000 7f36b154d640 1 -- 172.21.15.69:0/4151006760 learned_addr learned my addr 172.21.15.69:0/4151006760 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:36.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.667+0000 7f36b1d4e640 1 --2- 172.21.15.69:0/4151006760 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3698005910 0x7f36ac0b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:36.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.667+0000 7f36b254f640 1 --2- 172.21.15.69:0/4151006760 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f36ac157070 0x7f36ac0ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:36.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.667+0000 7f36b154d640 1 -- 172.21.15.69:0/4151006760 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f36ac157070 msgr2=0x7f36ac0ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:36.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.667+0000 7f36b154d640 1 --2- 172.21.15.69:0/4151006760 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f36ac157070 0x7f36ac0ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.667+0000 7f36b154d640 1 -- 172.21.15.69:0/4151006760 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3698005910 msgr2=0x7f36ac0b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:36.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.667+0000 7f36b154d640 1 --2- 172.21.15.69:0/4151006760 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3698005910 0x7f36ac0b46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.667+0000 7f36b154d640 1 -- 172.21.15.69:0/4151006760 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36ac1f6f60 con 0x7f36ac15f150 2024-04-25T11:00:36.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.667+0000 7f36b254f640 1 --2- 172.21.15.69:0/4151006760 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f36ac157070 0x7f36ac0ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:36.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.667+0000 7f36b1d4e640 1 --2- 172.21.15.69:0/4151006760 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3698005910 0x7f36ac0b46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:36.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.667+0000 7f36b154d640 1 --2- 172.21.15.69:0/4151006760 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36ac15f150 0x7f36ac0b4be0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f369c02ae20 tx=0x7f36ac1605f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:36.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.668+0000 7f36a27fc640 1 -- 172.21.15.69:0/4151006760 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369c042070 con 0x7f36ac15f150 2024-04-25T11:00:36.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.668+0000 7f36a27fc640 1 -- 172.21.15.69:0/4151006760 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f369c007480 con 0x7f36ac15f150 2024-04-25T11:00:36.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.668+0000 7f36b3fb2640 1 -- 172.21.15.69:0/4151006760 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36ac1f7240 con 0x7f36ac15f150 2024-04-25T11:00:36.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.668+0000 7f36a27fc640 1 -- 172.21.15.69:0/4151006760 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369c001cb0 con 0x7f36ac15f150 2024-04-25T11:00:36.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.668+0000 7f36b3fb2640 1 -- 172.21.15.69:0/4151006760 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36ac0cc410 con 0x7f36ac15f150 2024-04-25T11:00:36.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.669+0000 7f36b3fb2640 1 -- 172.21.15.69:0/4151006760 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36ac15f8c0 con 0x7f36ac15f150 2024-04-25T11:00:36.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.669+0000 7f36a27fc640 1 -- 172.21.15.69:0/4151006760 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f369c045070 con 0x7f36ac15f150 2024-04-25T11:00:36.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.670+0000 7f36a27fc640 1 --2- 172.21.15.69:0/4151006760 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f36900744b0 0x7f36900769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.670+0000 7f36a27fc640 1 -- 172.21.15.69:0/4151006760 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7f369c0b5f40 con 0x7f36ac15f150 2024-04-25T11:00:36.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.670+0000 7f36b1d4e640 1 --2- 172.21.15.69:0/4151006760 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f36900744b0 0x7f36900769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:36.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.670+0000 7f36b1d4e640 1 --2- 172.21.15.69:0/4151006760 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f36900744b0 0x7f36900769c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f36a8009670 tx=0x7f36a800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:36.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.675+0000 7f36a27fc640 1 -- 172.21.15.69:0/4151006760 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f369c04c030 con 0x7f36ac15f150 2024-04-25T11:00:36.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.850+0000 7f36b3fb2640 1 -- 172.21.15.69:0/4151006760 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f36ac15f610 con 0x7f36ac15f150 2024-04-25T11:00:36.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.851+0000 7f36a27fc640 1 -- 172.21.15.69:0/4151006760 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16798 (secure 0 0 0) 0x7f369c080790 con 0x7f36ac15f150 2024-04-25T11:00:36.852 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:00:36.852 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":32,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T11:00:30.066894+0000","last_up_change":"2024-04-25T10:59:05.546839+0000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:59:06.422674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:00:05.007593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T11:00:05.558880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6825","nonce":1876101065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6827","nonce":1876101065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6831","nonce":1876101065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6829","nonce":1876101065}]},"public_addr":"172.21.15.69:6825/1876101065","cluster_addr":"172.21.15.69:6827/1876101065","heartbeat_back_addr":"172.21.15.69:6831/1876101065","heartbeat_front_addr":"172.21.15.69:6829/1876101065","state":["exists","up"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6801","nonce":1806438563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6803","nonce":1806438563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6807","nonce":1806438563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6805","nonce":1806438563}]},"public_addr":"172.21.15.69:6801/1806438563","cluster_addr":"172.21.15.69:6803/1806438563","heartbeat_back_addr":"172.21.15.69:6807/1806438563","heartbeat_front_addr":"172.21.15.69:6805/1806438563","state":["exists","up"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6817","nonce":1593617983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6819","nonce":1593617983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6823","nonce":1593617983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6821","nonce":1593617983}]},"public_addr":"172.21.15.69:6817/1593617983","cluster_addr":"172.21.15.69:6819/1593617983","heartbeat_back_addr":"172.21.15.69:6823/1593617983","heartbeat_front_addr":"172.21.15.69:6821/1593617983","state":["exists","up"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6809","nonce":3950661294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6811","nonce":3950661294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6815","nonce":3950661294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6813","nonce":3950661294}]},"public_addr":"172.21.15.69:6809/3950661294","cluster_addr":"172.21.15.69:6811/3950661294","heartbeat_back_addr":"172.21.15.69:6815/3950661294","heartbeat_front_addr":"172.21.15.69:6813/3950661294","state":["exists","up"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6801","nonce":2508929872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6803","nonce":2508929872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6807","nonce":2508929872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6805","nonce":2508929872}]},"public_addr":"172.21.15.146:6801/2508929872","cluster_addr":"172.21.15.146:6803/2508929872","heartbeat_back_addr":"172.21.15.146:6807/2508929872","heartbeat_front_addr":"172.21.15.146:6805/2508929872","state":["exists","up"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6811","nonce":1310911138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6816","nonce":1310911138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6823","nonce":1310911138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6821","nonce":1310911138}]},"public_addr":"172.21.15.146:6811/1310911138","cluster_addr":"172.21.15.146:6816/1310911138","heartbeat_back_addr":"172.21.15.146:6823/1310911138","heartbeat_front_addr":"172.21.15.146:6821/1310911138","state":["exists","up"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6810","nonce":634688021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6813","nonce":634688021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6820","nonce":634688021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6817","nonce":634688021}]},"public_addr":"172.21.15.146:6810/634688021","cluster_addr":"172.21.15.146:6813/634688021","heartbeat_back_addr":"172.21.15.146:6820/634688021","heartbeat_front_addr":"172.21.15.146:6817/634688021","state":["exists","up"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6825","nonce":3814215935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6827","nonce":3814215935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6831","nonce":3814215935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6829","nonce":3814215935}]},"public_addr":"172.21.15.146:6825/3814215935","cluster_addr":"172.21.15.146:6827/3814215935","heartbeat_back_addr":"172.21.15.146:6831/3814215935","heartbeat_front_addr":"172.21.15.146:6829/3814215935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.820200+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.850189+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.764560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.866248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.807897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.869881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.782310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.827131+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6837/3153221129":"2024-04-26T11:00:30.066863+0000","172.21.15.69:6835/3153221129":"2024-04-26T11:00:30.066863+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:00:36.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.853+0000 7f36b3fb2640 1 -- 172.21.15.69:0/4151006760 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f36900744b0 msgr2=0x7f36900769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:36.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.853+0000 7f36b3fb2640 1 --2- 172.21.15.69:0/4151006760 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f36900744b0 0x7f36900769c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f36a8009670 tx=0x7f36a800b040 comp rx=0 tx=0).stop 2024-04-25T11:00:36.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.853+0000 7f36b3fb2640 1 -- 172.21.15.69:0/4151006760 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36ac15f150 msgr2=0x7f36ac0b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:36.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.853+0000 7f36b3fb2640 1 --2- 172.21.15.69:0/4151006760 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36ac15f150 0x7f36ac0b4be0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f369c02ae20 tx=0x7f36ac1605f0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.853+0000 7f36b3fb2640 1 -- 172.21.15.69:0/4151006760 shutdown_connections 2024-04-25T11:00:36.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.853+0000 7f36b3fb2640 1 --2- 172.21.15.69:0/4151006760 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f36900744b0 0x7f36900769c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.853+0000 7f36b3fb2640 1 --2- 172.21.15.69:0/4151006760 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f36ac157070 0x7f36ac0ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.853+0000 7f36b3fb2640 1 --2- 172.21.15.69:0/4151006760 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36ac15f150 0x7f36ac0b4be0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.853+0000 7f36b3fb2640 1 --2- 172.21.15.69:0/4151006760 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3698005910 0x7f36ac0b46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.853+0000 7f36b3fb2640 1 -- 172.21.15.69:0/4151006760 >> 172.21.15.69:0/4151006760 conn(0x7f36ac0d1ae0 msgr2=0x7f36ac0b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:36.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.853+0000 7f36b3fb2640 1 -- 172.21.15.69:0/4151006760 shutdown_connections 2024-04-25T11:00:36.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.853+0000 7f36b3fb2640 1 -- 172.21.15.69:0/4151006760 wait complete. 2024-04-25T11:00:36.865 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:59:06.422674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:00:05.007593+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '30', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:00:05.558880+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:00:36.865 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:00:36.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.975+0000 7f2af6ee9640 1 Processor -- start 2024-04-25T11:00:36.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.975+0000 7f2af6ee9640 1 -- start start 2024-04-25T11:00:36.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.976+0000 7f2af6ee9640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2af006dce0 0x7f2af00cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.976+0000 7f2af6ee9640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2af0071110 con 0x7f2af015adf0 2024-04-25T11:00:36.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.976+0000 7f2af6ee9640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2af006fb50 con 0x7f2af0070630 2024-04-25T11:00:36.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.976+0000 7f2af6ee9640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f2af00713f0 con 0x7f2af006dce0 2024-04-25T11:00:36.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.976+0000 7f2af5486640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f2af015adf0 0x7f2af0160d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:39424/0 (socket says 172.21.15.69:39424) 2024-04-25T11:00:36.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.976+0000 7f2af5486640 1 -- 172.21.15.69:0/3839958304 learned_addr learned my addr 172.21.15.69:0/3839958304 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:36.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.977+0000 7f2aeffff640 1 --2- 172.21.15.69:0/3839958304 >> v2:172.21.15.146:3300/0 conn(0x7f2af006dce0 0x7f2af00cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:36.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.977+0000 7f2aef7fe640 1 -- 172.21.15.69:0/3839958304 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1623598855 0 0) 0x7f2af0071110 con 0x7f2af015adf0 2024-04-25T11:00:36.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.977+0000 7f2aef7fe640 1 -- 172.21.15.69:0/3839958304 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ae0003520 con 0x7f2af015adf0 2024-04-25T11:00:36.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.977+0000 7f2aef7fe640 1 -- 172.21.15.69:0/3839958304 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 271451118 0 0) 0x7f2ae0003520 con 0x7f2af015adf0 2024-04-25T11:00:36.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.977+0000 7f2aeffff640 1 -- 172.21.15.69:0/3839958304 >> v1:172.21.15.146:6789/0 conn(0x7f2af0070630 legacy=0x7f2af006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:36.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.977+0000 7f2aeffff640 1 -- 172.21.15.69:0/3839958304 >> v1:172.21.15.69:6789/0 conn(0x7f2af015adf0 legacy=0x7f2af0160d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:36.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.977+0000 7f2aeffff640 1 -- 172.21.15.69:0/3839958304 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af006fe30 con 0x7f2af006dce0 2024-04-25T11:00:36.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.978+0000 7f2aeffff640 1 --2- 172.21.15.69:0/3839958304 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2af006dce0 0x7f2af00cec50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0097a0 tx=0x7f2adc02a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d9c3dd0c2cd9a81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:36.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.978+0000 7f2aef7fe640 1 -- 172.21.15.69:0/3839958304 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc037070 con 0x7f2af006dce0 2024-04-25T11:00:36.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.978+0000 7f2aef7fe640 1 -- 172.21.15.69:0/3839958304 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2adc02abc0 con 0x7f2af006dce0 2024-04-25T11:00:36.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.978+0000 7f2aef7fe640 1 -- 172.21.15.69:0/3839958304 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae80019f0 con 0x7f2af006dce0 2024-04-25T11:00:36.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.979+0000 7f2af6ee9640 1 -- 172.21.15.69:0/3839958304 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2af006dce0 msgr2=0x7f2af00cec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:36.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.979+0000 7f2af6ee9640 1 --2- 172.21.15.69:0/3839958304 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2af006dce0 0x7f2af00cec50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0097a0 tx=0x7f2adc02a130 comp rx=0 tx=0).stop 2024-04-25T11:00:36.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.979+0000 7f2af6ee9640 1 -- 172.21.15.69:0/3839958304 shutdown_connections 2024-04-25T11:00:36.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.979+0000 7f2af6ee9640 1 --2- 172.21.15.69:0/3839958304 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2af006dce0 0x7f2af00cec50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.979+0000 7f2af6ee9640 1 -- 172.21.15.69:0/3839958304 >> 172.21.15.69:0/3839958304 conn(0x7f2af0073090 msgr2=0x7f2af00734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:36.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.979+0000 7f2af6ee9640 1 -- 172.21.15.69:0/3839958304 shutdown_connections 2024-04-25T11:00:36.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.979+0000 7f2af6ee9640 1 -- 172.21.15.69:0/3839958304 wait complete. 2024-04-25T11:00:36.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.980+0000 7f2af6ee9640 1 Processor -- start 2024-04-25T11:00:36.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.981+0000 7f2af6ee9640 1 -- start start 2024-04-25T11:00:36.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.982+0000 7f2af6ee9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2af0070630 0x7f2af014af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.982+0000 7f2af6ee9640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2af015adf0 0x7f2af015c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.982+0000 7f2af6ee9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2af015cca0 0x7f2af015fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.982+0000 7f2af6ee9640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2af0070c80 con 0x7f2af015cca0 2024-04-25T11:00:36.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.982+0000 7f2af6ee9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2af00701a0 con 0x7f2af0070630 2024-04-25T11:00:36.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.982+0000 7f2af6ee9640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f2af0070e00 con 0x7f2af015adf0 2024-04-25T11:00:36.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.982+0000 7f2af5486640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2af015cca0 0x7f2af015fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:36.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.982+0000 7f2af5486640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2af015cca0 0x7f2af015fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47816/0 (socket says 172.21.15.69:47816) 2024-04-25T11:00:36.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.982+0000 7f2af5486640 1 -- 172.21.15.69:0/3145141791 learned_addr learned my addr 172.21.15.69:0/3145141791 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:36.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.982+0000 7f2af4c85640 1 --2- 172.21.15.69:0/3145141791 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2af0070630 0x7f2af014af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:36.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.982+0000 7f2aeffff640 1 --2- 172.21.15.69:0/3145141791 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2af015adf0 0x7f2af015c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:36.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.982+0000 7f2af5486640 1 -- 172.21.15.69:0/3145141791 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2af015adf0 msgr2=0x7f2af015c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:36.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.982+0000 7f2af5486640 1 --2- 172.21.15.69:0/3145141791 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2af015adf0 0x7f2af015c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.982+0000 7f2af5486640 1 -- 172.21.15.69:0/3145141791 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2af0070630 msgr2=0x7f2af014af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:36.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.982+0000 7f2af5486640 1 --2- 172.21.15.69:0/3145141791 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2af0070630 0x7f2af014af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:36.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.983+0000 7f2af5486640 1 -- 172.21.15.69:0/3145141791 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af0158d10 con 0x7f2af015cca0 2024-04-25T11:00:36.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.983+0000 7f2af5486640 1 --2- 172.21.15.69:0/3145141791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2af015cca0 0x7f2af015fa10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2ae800cc30 tx=0x7f2ae800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:36.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.983+0000 7f2aed7fa640 1 -- 172.21.15.69:0/3145141791 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8017070 con 0x7f2af015cca0 2024-04-25T11:00:36.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.983+0000 7f2aed7fa640 1 -- 172.21.15.69:0/3145141791 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ae800ad40 con 0x7f2af015cca0 2024-04-25T11:00:36.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.983+0000 7f2af6ee9640 1 -- 172.21.15.69:0/3145141791 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2af0156760 con 0x7f2af015cca0 2024-04-25T11:00:36.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.983+0000 7f2aed7fa640 1 -- 172.21.15.69:0/3145141791 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8012640 con 0x7f2af015cca0 2024-04-25T11:00:36.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.983+0000 7f2af6ee9640 1 -- 172.21.15.69:0/3145141791 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2af0156ce0 con 0x7f2af015cca0 2024-04-25T11:00:36.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.984+0000 7f2af6ee9640 1 -- 172.21.15.69:0/3145141791 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2af015d1e0 con 0x7f2af015cca0 2024-04-25T11:00:36.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.984+0000 7f2aed7fa640 1 -- 172.21.15.69:0/3145141791 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2ae801a070 con 0x7f2af015cca0 2024-04-25T11:00:36.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.985+0000 7f2aed7fa640 1 --2- 172.21.15.69:0/3145141791 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f2ad40744e0 0x7f2ad40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:36.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.985+0000 7f2aed7fa640 1 -- 172.21.15.69:0/3145141791 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7f2ae8095290 con 0x7f2af015cca0 2024-04-25T11:00:36.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.985+0000 7f2af4c85640 1 --2- 172.21.15.69:0/3145141791 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f2ad40744e0 0x7f2ad40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:36.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.985+0000 7f2af4c85640 1 --2- 172.21.15.69:0/3145141791 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f2ad40744e0 0x7f2ad40769f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4000f60 tx=0x7f2ae4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:36.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:36.989+0000 7f2aed7fa640 1 -- 172.21.15.69:0/3145141791 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2ae805f370 con 0x7f2af015cca0 2024-04-25T11:00:37.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.166+0000 7f2af6ee9640 1 -- 172.21.15.69:0/3145141791 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2af0050740 con 0x7f2af015cca0 2024-04-25T11:00:37.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.166+0000 7f2aed7fa640 1 -- 172.21.15.69:0/3145141791 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) ==== 93+0+10 (secure 0 0 0) 0x7f2ae8063670 con 0x7f2af015cca0 2024-04-25T11:00:37.167 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-25T11:00:37.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.168+0000 7f2af6ee9640 1 -- 172.21.15.69:0/3145141791 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f2ad40744e0 msgr2=0x7f2ad40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:37.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.168+0000 7f2af6ee9640 1 --2- 172.21.15.69:0/3145141791 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f2ad40744e0 0x7f2ad40769f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4000f60 tx=0x7f2ae4009040 comp rx=0 tx=0).stop 2024-04-25T11:00:37.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.168+0000 7f2af6ee9640 1 -- 172.21.15.69:0/3145141791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2af015cca0 msgr2=0x7f2af015fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:37.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.168+0000 7f2af6ee9640 1 --2- 172.21.15.69:0/3145141791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2af015cca0 0x7f2af015fa10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2ae800cc30 tx=0x7f2ae800a430 comp rx=0 tx=0).stop 2024-04-25T11:00:37.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.168+0000 7f2af6ee9640 1 -- 172.21.15.69:0/3145141791 shutdown_connections 2024-04-25T11:00:37.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.168+0000 7f2af6ee9640 1 --2- 172.21.15.69:0/3145141791 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f2ad40744e0 0x7f2ad40769f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:37.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.168+0000 7f2af6ee9640 1 --2- 172.21.15.69:0/3145141791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2af015cca0 0x7f2af015fa10 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:37.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.168+0000 7f2af6ee9640 1 --2- 172.21.15.69:0/3145141791 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2af015adf0 0x7f2af015c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:37.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.168+0000 7f2af6ee9640 1 --2- 172.21.15.69:0/3145141791 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2af0070630 0x7f2af014af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:37.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.168+0000 7f2af6ee9640 1 -- 172.21.15.69:0/3145141791 >> 172.21.15.69:0/3145141791 conn(0x7f2af0073090 msgr2=0x7f2af00d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:37.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.168+0000 7f2af6ee9640 1 -- 172.21.15.69:0/3145141791 shutdown_connections 2024-04-25T11:00:37.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.168+0000 7f2af6ee9640 1 -- 172.21.15.69:0/3145141791 wait complete. 2024-04-25T11:00:37.180 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:00:37.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.289+0000 7f777ab64640 1 Processor -- start 2024-04-25T11:00:37.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.289+0000 7f777ab64640 1 -- start start 2024-04-25T11:00:37.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.290+0000 7f777ab64640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f777408d630 0x7f77740b90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:37.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.290+0000 7f777ab64640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77740d21e0 con 0x7f777408dec0 2024-04-25T11:00:37.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.290+0000 7f777ab64640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77740d1940 con 0x7f77740b1c50 2024-04-25T11:00:37.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.290+0000 7f777ab64640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f77740b95f0 con 0x7f777408d630 2024-04-25T11:00:37.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.290+0000 7f7778900640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f777408dec0 0x7f777408cf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:39440/0 (socket says 172.21.15.69:39440) 2024-04-25T11:00:37.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.290+0000 7f7778900640 1 -- 172.21.15.69:0/1788239088 learned_addr learned my addr 172.21.15.69:0/1788239088 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:37.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.291+0000 7f7773fff640 1 --2- 172.21.15.69:0/1788239088 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f777408d630 0x7f77740b90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:37.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.291+0000 7f77737fe640 1 -- 172.21.15.69:0/1788239088 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2386452605 0 0) 0x7f77740d21e0 con 0x7f777408dec0 2024-04-25T11:00:37.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.291+0000 7f77737fe640 1 -- 172.21.15.69:0/1788239088 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7764003520 con 0x7f777408dec0 2024-04-25T11:00:37.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.291+0000 7f77737fe640 1 -- 172.21.15.69:0/1788239088 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 601450699 0 0) 0x7f7764003520 con 0x7f777408dec0 2024-04-25T11:00:37.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.291+0000 7f77737fe640 1 -- 172.21.15.69:0/1788239088 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f777408d630 msgr2=0x7f77740b90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:37.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.291+0000 7f77737fe640 1 --2- 172.21.15.69:0/1788239088 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f777408d630 0x7f77740b90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:37.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.291+0000 7f77737fe640 1 -- 172.21.15.69:0/1788239088 >> v1:172.21.15.146:6789/0 conn(0x7f77740b1c50 legacy=0x7f77740ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:37.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.291+0000 7f77737fe640 1 -- 172.21.15.69:0/1788239088 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77740d1c20 con 0x7f777408dec0 2024-04-25T11:00:37.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.291+0000 7f77737fe640 1 -- 172.21.15.69:0/1788239088 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7f7768001670 con 0x7f777408dec0 2024-04-25T11:00:37.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.291+0000 7f77737fe640 1 -- 172.21.15.69:0/1788239088 >> v1:172.21.15.69:6789/0 conn(0x7f777408dec0 legacy=0x7f777408cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:37.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.291+0000 7f77737fe640 1 --2- 172.21.15.69:0/1788239088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77640039e0 0x7f7764003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:37.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.291+0000 7f77737fe640 1 --2- 172.21.15.69:0/1788239088 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7764004800 0x7f7764004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:37.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.291+0000 7f77737fe640 1 --2- 172.21.15.69:0/1788239088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7764005150 0x7f776400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:37.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.291+0000 7f77737fe640 1 -- 172.21.15.69:0/1788239088 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f77740b95f0 con 0x7f7764005150 2024-04-25T11:00:37.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.291+0000 7f77737fe640 1 -- 172.21.15.69:0/1788239088 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f776400bf60 con 0x7f77640039e0 2024-04-25T11:00:37.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.291+0000 7f77737fe640 1 -- 172.21.15.69:0/1788239088 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f776400c160 con 0x7f7764004800 2024-04-25T11:00:37.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.291+0000 7f7779101640 1 --2- 172.21.15.69:0/1788239088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7764005150 0x7f776400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:37.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.291+0000 7f7778900640 1 --2- 172.21.15.69:0/1788239088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77640039e0 0x7f7764003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:37.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.291+0000 7f7773fff640 1 --2- 172.21.15.69:0/1788239088 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7764004800 0x7f7764004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:37.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.291+0000 7f7779101640 1 -- 172.21.15.69:0/1788239088 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7764004800 msgr2=0x7f7764004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:37.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.291+0000 7f7779101640 1 --2- 172.21.15.69:0/1788239088 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7764004800 0x7f7764004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:37.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.291+0000 7f7779101640 1 -- 172.21.15.69:0/1788239088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77640039e0 msgr2=0x7f7764003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:37.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.291+0000 7f7779101640 1 --2- 172.21.15.69:0/1788239088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77640039e0 0x7f7764003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:37.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.291+0000 7f7779101640 1 -- 172.21.15.69:0/1788239088 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7764003520 con 0x7f7764005150 2024-04-25T11:00:37.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.292+0000 7f7779101640 1 --2- 172.21.15.69:0/1788239088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7764005150 0x7f776400b9c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f776c001e20 tx=0x7f7764005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7670d8adfb812c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:37.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.292+0000 7f77737fe640 1 -- 172.21.15.69:0/1788239088 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c001a70 con 0x7f7764005150 2024-04-25T11:00:37.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.292+0000 7f77737fe640 1 -- 172.21.15.69:0/1788239088 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f776c02ae40 con 0x7f7764005150 2024-04-25T11:00:37.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.292+0000 7f77737fe640 1 -- 172.21.15.69:0/1788239088 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c003bb0 con 0x7f7764005150 2024-04-25T11:00:37.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.293+0000 7f777ab64640 1 -- 172.21.15.69:0/1788239088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7764005150 msgr2=0x7f776400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:37.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.293+0000 7f777ab64640 1 --2- 172.21.15.69:0/1788239088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7764005150 0x7f776400b9c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f776c001e20 tx=0x7f7764005830 comp rx=0 tx=0).stop 2024-04-25T11:00:37.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.293+0000 7f7779101640 1 -- 172.21.15.69:0/1788239088 reap_dead start 2024-04-25T11:00:37.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.293+0000 7f777ab64640 1 -- 172.21.15.69:0/1788239088 shutdown_connections 2024-04-25T11:00:37.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.293+0000 7f777ab64640 1 -- 172.21.15.69:0/1788239088 >> 172.21.15.69:0/1788239088 conn(0x7f77740d0510 msgr2=0x7f77740d0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:37.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.293+0000 7f777ab64640 1 -- 172.21.15.69:0/1788239088 shutdown_connections 2024-04-25T11:00:37.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.293+0000 7f777ab64640 1 -- 172.21.15.69:0/1788239088 wait complete. 2024-04-25T11:00:37.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.295+0000 7f777ab64640 1 Processor -- start 2024-04-25T11:00:37.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.295+0000 7f777ab64640 1 -- start start 2024-04-25T11:00:37.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.296+0000 7f777ab64640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7774150b20 0x7f777415b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:37.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.296+0000 7f777ab64640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f777415bee0 0x7f777415f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:37.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.296+0000 7f777ab64640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7774150f30 0x7f777415f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:37.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.296+0000 7f777ab64640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f77740d15d0 con 0x7f777415bee0 2024-04-25T11:00:37.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.296+0000 7f777ab64640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f77740b0720 con 0x7f7774150f30 2024-04-25T11:00:37.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.296+0000 7f777ab64640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f777408d630 con 0x7f7774150b20 2024-04-25T11:00:37.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.296+0000 7f7773fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f777415bee0 0x7f777415f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:37.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.296+0000 7f7773fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f777415bee0 0x7f777415f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47830/0 (socket says 172.21.15.69:47830) 2024-04-25T11:00:37.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.296+0000 7f7773fff640 1 -- 172.21.15.69:0/36177170 learned_addr learned my addr 172.21.15.69:0/36177170 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:37.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.296+0000 7f7779101640 1 --2- 172.21.15.69:0/36177170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7774150f30 0x7f777415f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:37.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.296+0000 7f7778900640 1 --2- 172.21.15.69:0/36177170 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7774150b20 0x7f777415b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:37.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.296+0000 7f7773fff640 1 -- 172.21.15.69:0/36177170 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7774150b20 msgr2=0x7f777415b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:37.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.296+0000 7f7773fff640 1 --2- 172.21.15.69:0/36177170 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7774150b20 0x7f777415b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:37.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.296+0000 7f7773fff640 1 -- 172.21.15.69:0/36177170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7774150f30 msgr2=0x7f777415f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:37.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.296+0000 7f7773fff640 1 --2- 172.21.15.69:0/36177170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7774150f30 0x7f777415f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:37.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.297+0000 7f7773fff640 1 -- 172.21.15.69:0/36177170 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7774151790 con 0x7f777415bee0 2024-04-25T11:00:37.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.297+0000 7f7773fff640 1 --2- 172.21.15.69:0/36177170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f777415bee0 0x7f777415f240 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f77600119a0 tx=0x7f7760011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:37.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.297+0000 7f77717fa640 1 -- 172.21.15.69:0/36177170 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776000faf0 con 0x7f777415bee0 2024-04-25T11:00:37.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.297+0000 7f77717fa640 1 -- 172.21.15.69:0/36177170 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f776000fc90 con 0x7f777415bee0 2024-04-25T11:00:37.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.297+0000 7f77717fa640 1 -- 172.21.15.69:0/36177170 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7760014440 con 0x7f777415bee0 2024-04-25T11:00:37.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.297+0000 7f777ab64640 1 -- 172.21.15.69:0/36177170 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f777415a900 con 0x7f777415bee0 2024-04-25T11:00:37.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.297+0000 7f777ab64640 1 -- 172.21.15.69:0/36177170 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f777415aec0 con 0x7f777415bee0 2024-04-25T11:00:37.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.298+0000 7f777ab64640 1 -- 172.21.15.69:0/36177170 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7774050740 con 0x7f777415bee0 2024-04-25T11:00:37.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.299+0000 7f77717fa640 1 -- 172.21.15.69:0/36177170 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7760012040 con 0x7f777415bee0 2024-04-25T11:00:37.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.299+0000 7f77717fa640 1 --2- 172.21.15.69:0/36177170 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f77580744b0 0x7f77580769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:37.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.299+0000 7f77717fa640 1 -- 172.21.15.69:0/36177170 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7f7760095810 con 0x7f777415bee0 2024-04-25T11:00:37.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.299+0000 7f7778900640 1 --2- 172.21.15.69:0/36177170 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f77580744b0 0x7f77580769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:37.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.300+0000 7f7778900640 1 --2- 172.21.15.69:0/36177170 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f77580744b0 0x7f77580769c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f777415f1d0 tx=0x7f7768009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:37.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.304+0000 7f77717fa640 1 -- 172.21.15.69:0/36177170 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7774151790 con 0x7f777415bee0 2024-04-25T11:00:37.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.476+0000 7f777ab64640 1 -- 172.21.15.69:0/36177170 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f77740cfed0 con 0x7f777415bee0 2024-04-25T11:00:37.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.476+0000 7f77717fa640 1 -- 172.21.15.69:0/36177170 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v32) ==== 104+0+11 (secure 0 0 0) 0x7f7760019030 con 0x7f777415bee0 2024-04-25T11:00:37.477 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-25T11:00:37.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.478+0000 7f777ab64640 1 -- 172.21.15.69:0/36177170 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f77580744b0 msgr2=0x7f77580769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:37.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.478+0000 7f777ab64640 1 --2- 172.21.15.69:0/36177170 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f77580744b0 0x7f77580769c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f777415f1d0 tx=0x7f7768009040 comp rx=0 tx=0).stop 2024-04-25T11:00:37.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.478+0000 7f777ab64640 1 -- 172.21.15.69:0/36177170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f777415bee0 msgr2=0x7f777415f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:37.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.478+0000 7f777ab64640 1 --2- 172.21.15.69:0/36177170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f777415bee0 0x7f777415f240 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f77600119a0 tx=0x7f7760011e60 comp rx=0 tx=0).stop 2024-04-25T11:00:37.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.479+0000 7f777ab64640 1 -- 172.21.15.69:0/36177170 shutdown_connections 2024-04-25T11:00:37.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.479+0000 7f777ab64640 1 --2- 172.21.15.69:0/36177170 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f77580744b0 0x7f77580769c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:37.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.479+0000 7f777ab64640 1 --2- 172.21.15.69:0/36177170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7774150f30 0x7f777415f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:37.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.479+0000 7f777ab64640 1 --2- 172.21.15.69:0/36177170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f777415bee0 0x7f777415f240 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:37.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.479+0000 7f777ab64640 1 --2- 172.21.15.69:0/36177170 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7774150b20 0x7f777415b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:37.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.479+0000 7f777ab64640 1 -- 172.21.15.69:0/36177170 >> 172.21.15.69:0/36177170 conn(0x7f77740d0510 msgr2=0x7f77740684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:37.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.479+0000 7f777ab64640 1 -- 172.21.15.69:0/36177170 shutdown_connections 2024-04-25T11:00:37.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.479+0000 7f777ab64640 1 -- 172.21.15.69:0/36177170 wait complete. 2024-04-25T11:00:37.490 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:00:37.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.598+0000 7f184a95f640 1 Processor -- start 2024-04-25T11:00:37.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.598+0000 7f184a95f640 1 -- start start 2024-04-25T11:00:37.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.600+0000 7f184a95f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18440b77d0 0x7f1844164bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:37.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.600+0000 7f184a95f640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f18440d0d60 0x7f1844160360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:37.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.600+0000 7f184a95f640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18440cffa0 con 0x7f18441608a0 2024-04-25T11:00:37.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.600+0000 7f184a95f640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f18440b7be0 con 0x7f18440d0d60 2024-04-25T11:00:37.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.600+0000 7f184a95f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1844165110 con 0x7f18440b77d0 2024-04-25T11:00:37.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.600+0000 7f18437fe640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f18440d0d60 0x7f1844160360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:37.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.600+0000 7f1843fff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18440b77d0 0x7f1844164bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:37.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.600+0000 7f1848efc640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f18441608a0 0x7f18440b2f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.69:44264/0 (socket says 172.21.15.69:44264) 2024-04-25T11:00:37.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.600+0000 7f18437fe640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f18440d0d60 0x7f1844160360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.69:48712/0 (socket says 172.21.15.69:48712) 2024-04-25T11:00:37.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.600+0000 7f1848efc640 1 -- 172.21.15.69:0/761707053 learned_addr learned my addr 172.21.15.69:0/761707053 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:37.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.600+0000 7f1843fff640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18440b77d0 0x7f1844164bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.69:52894/0 (socket says 172.21.15.69:52894) 2024-04-25T11:00:37.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.600+0000 7f1843fff640 1 -- 172.21.15.69:0/761707053 >> v2:172.21.15.146:3300/0 conn(0x7f18440d0d60 msgr2=0x7f1844160360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:37.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.600+0000 7f1843fff640 1 --2- 172.21.15.69:0/761707053 >> v2:172.21.15.146:3300/0 conn(0x7f18440d0d60 0x7f1844160360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:37.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.600+0000 7f1843fff640 1 -- 172.21.15.69:0/761707053 >> v1:172.21.15.146:6789/0 conn(0x7f18441608a0 legacy=0x7f18440b2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:37.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.600+0000 7f1843fff640 1 -- 172.21.15.69:0/761707053 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18440d0280 con 0x7f18440b77d0 2024-04-25T11:00:37.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.601+0000 7f1843fff640 1 --2- 172.21.15.69:0/761707053 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18440b77d0 0x7f1844164bd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f18340097a0 tx=0x7f183402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfe583616a95b0b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:37.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.601+0000 7f1842ffd640 1 -- 172.21.15.69:0/761707053 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834037070 con 0x7f18440b77d0 2024-04-25T11:00:37.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.601+0000 7f1842ffd640 1 -- 172.21.15.69:0/761707053 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1834032440 con 0x7f18440b77d0 2024-04-25T11:00:37.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.601+0000 7f1842ffd640 1 -- 172.21.15.69:0/761707053 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834032810 con 0x7f18440b77d0 2024-04-25T11:00:37.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.602+0000 7f184a95f640 1 -- 172.21.15.69:0/761707053 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18440b77d0 msgr2=0x7f1844164bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:37.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.602+0000 7f184a95f640 1 --2- 172.21.15.69:0/761707053 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18440b77d0 0x7f1844164bd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f18340097a0 tx=0x7f183402a210 comp rx=0 tx=0).stop 2024-04-25T11:00:37.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.602+0000 7f184a95f640 1 -- 172.21.15.69:0/761707053 shutdown_connections 2024-04-25T11:00:37.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.602+0000 7f184a95f640 1 --2- 172.21.15.69:0/761707053 >> v2:172.21.15.146:3300/0 conn(0x7f18440d0d60 0x7f1844160360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:37.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.602+0000 7f184a95f640 1 --2- 172.21.15.69:0/761707053 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18440b77d0 0x7f1844164bd0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:37.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.602+0000 7f184a95f640 1 -- 172.21.15.69:0/761707053 >> 172.21.15.69:0/761707053 conn(0x7f184409b7d0 msgr2=0x7f18440b70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:37.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.602+0000 7f184a95f640 1 -- 172.21.15.69:0/761707053 shutdown_connections 2024-04-25T11:00:37.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.602+0000 7f184a95f640 1 -- 172.21.15.69:0/761707053 wait complete. 2024-04-25T11:00:37.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.604+0000 7f184a95f640 1 Processor -- start 2024-04-25T11:00:37.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.604+0000 7f184a95f640 1 -- start start 2024-04-25T11:00:37.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.605+0000 7f184a95f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18440b77d0 0x7f1844162f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:37.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.605+0000 7f184a95f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18440d0d60 0x7f184406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:37.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.605+0000 7f184a95f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f184409daf0 0x7f184406d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:37.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.605+0000 7f184a95f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f18440d1170 con 0x7f18440d0d60 2024-04-25T11:00:37.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.605+0000 7f184a95f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f18440d08d0 con 0x7f18440b77d0 2024-04-25T11:00:37.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.605+0000 7f184a95f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f18440d12f0 con 0x7f184409daf0 2024-04-25T11:00:37.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.605+0000 7f18437fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18440d0d60 0x7f184406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:37.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.605+0000 7f18437fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18440d0d60 0x7f184406d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47840/0 (socket says 172.21.15.69:47840) 2024-04-25T11:00:37.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.605+0000 7f18437fe640 1 -- 172.21.15.69:0/1335347725 learned_addr learned my addr 172.21.15.69:0/1335347725 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:37.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.606+0000 7f1843fff640 1 --2- 172.21.15.69:0/1335347725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18440b77d0 0x7f1844162f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:37.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.606+0000 7f1848efc640 1 --2- 172.21.15.69:0/1335347725 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f184409daf0 0x7f184406d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:37.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.606+0000 7f18437fe640 1 -- 172.21.15.69:0/1335347725 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f184409daf0 msgr2=0x7f184406d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:37.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.606+0000 7f18437fe640 1 --2- 172.21.15.69:0/1335347725 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f184409daf0 0x7f184406d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:37.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.606+0000 7f18437fe640 1 -- 172.21.15.69:0/1335347725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18440b77d0 msgr2=0x7f1844162f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:37.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.606+0000 7f18437fe640 1 --2- 172.21.15.69:0/1335347725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18440b77d0 0x7f1844162f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:37.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.606+0000 7f18437fe640 1 -- 172.21.15.69:0/1335347725 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1844163470 con 0x7f18440d0d60 2024-04-25T11:00:37.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.606+0000 7f18437fe640 1 --2- 172.21.15.69:0/1335347725 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18440d0d60 0x7f184406d260 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f183800e870 tx=0x7f183800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:37.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.606+0000 7f1840ff9640 1 -- 172.21.15.69:0/1335347725 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838017070 con 0x7f18440d0d60 2024-04-25T11:00:37.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.606+0000 7f1840ff9640 1 -- 172.21.15.69:0/1335347725 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f183800ee60 con 0x7f18440d0d60 2024-04-25T11:00:37.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.606+0000 7f184a95f640 1 -- 172.21.15.69:0/1335347725 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1844160940 con 0x7f18440d0d60 2024-04-25T11:00:37.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.606+0000 7f1840ff9640 1 -- 172.21.15.69:0/1335347725 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183800cb60 con 0x7f18440d0d60 2024-04-25T11:00:37.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.606+0000 7f184a95f640 1 -- 172.21.15.69:0/1335347725 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1844160f00 con 0x7f18440d0d60 2024-04-25T11:00:37.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.607+0000 7f184a95f640 1 -- 172.21.15.69:0/1335347725 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f184406e2b0 con 0x7f18440d0d60 2024-04-25T11:00:37.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.607+0000 7f1840ff9640 1 -- 172.21.15.69:0/1335347725 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f183801a070 con 0x7f18440d0d60 2024-04-25T11:00:37.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.608+0000 7f1840ff9640 1 --2- 172.21.15.69:0/1335347725 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f18300744b0 0x7f18300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:37.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.608+0000 7f1843fff640 1 --2- 172.21.15.69:0/1335347725 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f18300744b0 0x7f18300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:37.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.608+0000 7f1840ff9640 1 -- 172.21.15.69:0/1335347725 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7f1838095060 con 0x7f18440d0d60 2024-04-25T11:00:37.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.608+0000 7f1843fff640 1 --2- 172.21.15.69:0/1335347725 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f18300744b0 0x7f18300769c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1834034000 tx=0x7f183402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:37.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.613+0000 7f1840ff9640 1 -- 172.21.15.69:0/1335347725 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f183805f820 con 0x7f18440d0d60 2024-04-25T11:00:37.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.786+0000 7f184a95f640 1 -- 172.21.15.69:0/1335347725 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1844050740 con 0x7f18440d0d60 2024-04-25T11:00:37.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.786+0000 7f1840ff9640 1 -- 172.21.15.69:0/1335347725 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v32) ==== 100+0+11 (secure 0 0 0) 0x7f183805f120 con 0x7f18440d0d60 2024-04-25T11:00:37.787 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-25T11:00:37.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.788+0000 7f184a95f640 1 -- 172.21.15.69:0/1335347725 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f18300744b0 msgr2=0x7f18300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:37.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.788+0000 7f184a95f640 1 --2- 172.21.15.69:0/1335347725 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f18300744b0 0x7f18300769c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1834034000 tx=0x7f183402e000 comp rx=0 tx=0).stop 2024-04-25T11:00:37.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.788+0000 7f184a95f640 1 -- 172.21.15.69:0/1335347725 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18440d0d60 msgr2=0x7f184406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:37.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.788+0000 7f184a95f640 1 --2- 172.21.15.69:0/1335347725 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18440d0d60 0x7f184406d260 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f183800e870 tx=0x7f183800ed30 comp rx=0 tx=0).stop 2024-04-25T11:00:37.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.788+0000 7f184a95f640 1 -- 172.21.15.69:0/1335347725 shutdown_connections 2024-04-25T11:00:37.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.788+0000 7f184a95f640 1 --2- 172.21.15.69:0/1335347725 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f18300744b0 0x7f18300769c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:37.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.788+0000 7f184a95f640 1 --2- 172.21.15.69:0/1335347725 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f184409daf0 0x7f184406d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:37.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.788+0000 7f184a95f640 1 --2- 172.21.15.69:0/1335347725 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18440d0d60 0x7f184406d260 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:37.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.788+0000 7f184a95f640 1 --2- 172.21.15.69:0/1335347725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18440b77d0 0x7f1844162f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:37.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.788+0000 7f184a95f640 1 -- 172.21.15.69:0/1335347725 >> 172.21.15.69:0/1335347725 conn(0x7f184409b7d0 msgr2=0x7f184406fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:37.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.788+0000 7f184a95f640 1 -- 172.21.15.69:0/1335347725 shutdown_connections 2024-04-25T11:00:37.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.788+0000 7f184a95f640 1 -- 172.21.15.69:0/1335347725 wait complete. 2024-04-25T11:00:37.800 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:00:37.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.906+0000 7fed9ae6e640 1 Processor -- start 2024-04-25T11:00:37.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.906+0000 7fed9ae6e640 1 -- start start 2024-04-25T11:00:37.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.907+0000 7fed9ae6e640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fed94160510 0x7fed9406f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:37.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.907+0000 7fed9ae6e640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed940cce10 con 0x7fed9406f910 2024-04-25T11:00:37.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.907+0000 7fed9ae6e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed940cd0b0 con 0x7fed941645e0 2024-04-25T11:00:37.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.907+0000 7fed9ae6e640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fed94164aa0 con 0x7fed94160510 2024-04-25T11:00:37.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.907+0000 7fed98c0a640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fed94160510 0x7fed9406f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:37.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.907+0000 7fed98c0a640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fed94160510 0x7fed9406f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47844/0 (socket says 172.21.15.69:47844) 2024-04-25T11:00:37.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.907+0000 7fed98c0a640 1 -- 172.21.15.69:0/629747086 learned_addr learned my addr 172.21.15.69:0/629747086 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:37.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.907+0000 7fed93fff640 1 --1- 172.21.15.69:0/629747086 >> v1:172.21.15.69:6789/0 conn(0x7fed9406f910 0x7fed94163ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:39444/0 (socket says 172.21.15.69:39444) 2024-04-25T11:00:37.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.908+0000 7fed98c0a640 1 -- 172.21.15.69:0/629747086 >> v1:172.21.15.146:6789/0 conn(0x7fed941645e0 legacy=0x7fed9406d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:37.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.908+0000 7fed98c0a640 1 -- 172.21.15.69:0/629747086 >> v1:172.21.15.69:6789/0 conn(0x7fed9406f910 legacy=0x7fed94163ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:37.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.908+0000 7fed937fe640 1 -- 172.21.15.69:0/629747086 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1662604009 0 0) 0x7fed940cce10 con 0x7fed9406f910 2024-04-25T11:00:37.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.908+0000 7fed98c0a640 1 -- 172.21.15.69:0/629747086 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed94164d80 con 0x7fed94160510 2024-04-25T11:00:37.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.908+0000 7fed98c0a640 1 --2- 172.21.15.69:0/629747086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed94160510 0x7fed9406f3d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fed88009870 tx=0x7fed8802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212895105a02ebef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:37.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.909+0000 7fed937fe640 1 -- 172.21.15.69:0/629747086 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed88037070 con 0x7fed94160510 2024-04-25T11:00:37.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.909+0000 7fed937fe640 1 -- 172.21.15.69:0/629747086 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed8802ae50 con 0x7fed94160510 2024-04-25T11:00:37.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.909+0000 7fed937fe640 1 -- 172.21.15.69:0/629747086 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed88032860 con 0x7fed94160510 2024-04-25T11:00:37.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.909+0000 7fed9ae6e640 1 -- 172.21.15.69:0/629747086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed94160510 msgr2=0x7fed9406f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:37.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.909+0000 7fed9ae6e640 1 --2- 172.21.15.69:0/629747086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed94160510 0x7fed9406f3d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fed88009870 tx=0x7fed8802a300 comp rx=0 tx=0).stop 2024-04-25T11:00:37.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.909+0000 7fed9ae6e640 1 -- 172.21.15.69:0/629747086 shutdown_connections 2024-04-25T11:00:37.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.909+0000 7fed9ae6e640 1 --2- 172.21.15.69:0/629747086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed94160510 0x7fed9406f3d0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:37.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.909+0000 7fed9ae6e640 1 -- 172.21.15.69:0/629747086 >> 172.21.15.69:0/629747086 conn(0x7fed940ce790 msgr2=0x7fed940ceba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:37.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.909+0000 7fed9ae6e640 1 -- 172.21.15.69:0/629747086 shutdown_connections 2024-04-25T11:00:37.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.909+0000 7fed9ae6e640 1 -- 172.21.15.69:0/629747086 wait complete. 2024-04-25T11:00:37.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.911+0000 7fed9ae6e640 1 Processor -- start 2024-04-25T11:00:37.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.911+0000 7fed9ae6e640 1 -- start start 2024-04-25T11:00:37.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.912+0000 7fed9ae6e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed9406f910 0x7fed941f69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:37.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.912+0000 7fed9ae6e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed94160510 0x7fed940b6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:37.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.913+0000 7fed9ae6e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed940c9240 0x7fed940b6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:37.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.913+0000 7fed9ae6e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fed9406dde0 con 0x7fed940c9240 2024-04-25T11:00:37.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.913+0000 7fed9ae6e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fed9406dc60 con 0x7fed9406f910 2024-04-25T11:00:37.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.913+0000 7fed9ae6e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fed940d1780 con 0x7fed94160510 2024-04-25T11:00:37.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.913+0000 7fed9940b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed940c9240 0x7fed940b6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:37.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.913+0000 7fed9940b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed940c9240 0x7fed940b6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47848/0 (socket says 172.21.15.69:47848) 2024-04-25T11:00:37.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.913+0000 7fed9940b640 1 -- 172.21.15.69:0/1786549943 learned_addr learned my addr 172.21.15.69:0/1786549943 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:37.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.913+0000 7fed93fff640 1 --2- 172.21.15.69:0/1786549943 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed94160510 0x7fed940b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:37.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.913+0000 7fed98c0a640 1 --2- 172.21.15.69:0/1786549943 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed9406f910 0x7fed941f69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:37.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.913+0000 7fed9940b640 1 -- 172.21.15.69:0/1786549943 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed94160510 msgr2=0x7fed940b6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:37.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.913+0000 7fed9940b640 1 --2- 172.21.15.69:0/1786549943 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed94160510 0x7fed940b6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:37.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.913+0000 7fed9940b640 1 -- 172.21.15.69:0/1786549943 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed9406f910 msgr2=0x7fed941f69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:37.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.913+0000 7fed9940b640 1 --2- 172.21.15.69:0/1786549943 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed9406f910 0x7fed941f69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:37.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.913+0000 7fed9940b640 1 -- 172.21.15.69:0/1786549943 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed940a01b0 con 0x7fed940c9240 2024-04-25T11:00:37.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.913+0000 7fed9940b640 1 --2- 172.21.15.69:0/1786549943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed940c9240 0x7fed940b6a60 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fed8c00c730 tx=0x7fed8c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:37.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.913+0000 7fed917fa640 1 -- 172.21.15.69:0/1786549943 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8c003b70 con 0x7fed940c9240 2024-04-25T11:00:37.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.913+0000 7fed917fa640 1 -- 172.21.15.69:0/1786549943 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed8c003d10 con 0x7fed940c9240 2024-04-25T11:00:37.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.913+0000 7fed9ae6e640 1 -- 172.21.15.69:0/1786549943 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed940a0490 con 0x7fed940c9240 2024-04-25T11:00:37.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.913+0000 7fed917fa640 1 -- 172.21.15.69:0/1786549943 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8c005310 con 0x7fed940c9240 2024-04-25T11:00:37.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.913+0000 7fed9ae6e640 1 -- 172.21.15.69:0/1786549943 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed940a1ff0 con 0x7fed940c9240 2024-04-25T11:00:37.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.914+0000 7fed9ae6e640 1 -- 172.21.15.69:0/1786549943 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed940710e0 con 0x7fed940c9240 2024-04-25T11:00:37.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.915+0000 7fed917fa640 1 -- 172.21.15.69:0/1786549943 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fed8c0054b0 con 0x7fed940c9240 2024-04-25T11:00:37.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.915+0000 7fed917fa640 1 --2- 172.21.15.69:0/1786549943 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fed780744e0 0x7fed780769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:37.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.916+0000 7fed917fa640 1 -- 172.21.15.69:0/1786549943 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7fed8c094f00 con 0x7fed940c9240 2024-04-25T11:00:37.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.916+0000 7fed98c0a640 1 --2- 172.21.15.69:0/1786549943 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fed780744e0 0x7fed780769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:37.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.916+0000 7fed98c0a640 1 --2- 172.21.15.69:0/1786549943 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fed780744e0 0x7fed780769f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fed88034000 tx=0x7fed8802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:37.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:37.920+0000 7fed917fa640 1 -- 172.21.15.69:0/1786549943 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fed940710e0 con 0x7fed940c9240 2024-04-25T11:00:38.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.113+0000 7fed9ae6e640 1 -- 172.21.15.69:0/1786549943 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fed9406f700 con 0x7fed940c9240 2024-04-25T11:00:38.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.114+0000 7fed917fa640 1 -- 172.21.15.69:0/1786549943 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v32) ==== 80+0+122 (secure 0 0 0) 0x7fed9406f700 con 0x7fed940c9240 2024-04-25T11:00:38.114 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6837/3153221129 2024-04-26T11:00:30.066863+0000 2024-04-25T11:00:38.115 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6835/3153221129 2024-04-26T11:00:30.066863+0000 2024-04-25T11:00:38.115 INFO:teuthology.orchestra.run.smithi069.stderr:listed 2 entries 2024-04-25T11:00:38.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.116+0000 7fed9ae6e640 1 -- 172.21.15.69:0/1786549943 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fed780744e0 msgr2=0x7fed780769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:38.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.116+0000 7fed9ae6e640 1 --2- 172.21.15.69:0/1786549943 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fed780744e0 0x7fed780769f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fed88034000 tx=0x7fed8802e000 comp rx=0 tx=0).stop 2024-04-25T11:00:38.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.116+0000 7fed9ae6e640 1 -- 172.21.15.69:0/1786549943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed940c9240 msgr2=0x7fed940b6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:38.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.116+0000 7fed9ae6e640 1 --2- 172.21.15.69:0/1786549943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed940c9240 0x7fed940b6a60 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fed8c00c730 tx=0x7fed8c00cbf0 comp rx=0 tx=0).stop 2024-04-25T11:00:38.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.116+0000 7fed9ae6e640 1 -- 172.21.15.69:0/1786549943 shutdown_connections 2024-04-25T11:00:38.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.116+0000 7fed9ae6e640 1 --2- 172.21.15.69:0/1786549943 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fed780744e0 0x7fed780769f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:38.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.116+0000 7fed9ae6e640 1 --2- 172.21.15.69:0/1786549943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed940c9240 0x7fed940b6a60 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:38.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.116+0000 7fed9ae6e640 1 --2- 172.21.15.69:0/1786549943 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed94160510 0x7fed940b6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:38.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.116+0000 7fed9ae6e640 1 --2- 172.21.15.69:0/1786549943 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed9406f910 0x7fed941f69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:38.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.116+0000 7fed9ae6e640 1 -- 172.21.15.69:0/1786549943 >> 172.21.15.69:0/1786549943 conn(0x7fed940ce790 msgr2=0x7fed9406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:38.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.116+0000 7fed9ae6e640 1 -- 172.21.15.69:0/1786549943 shutdown_connections 2024-04-25T11:00:38.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.116+0000 7fed9ae6e640 1 -- 172.21.15.69:0/1786549943 wait complete. 2024-04-25T11:00:38.128 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:00:38.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:00:38.128 DEBUG:teuthology.orchestra.run.smithi069:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T11:00:38.154 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi069.front.sepia.ceph.com 2024-04-25T11:00:38.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:00:38.154 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:00:38.209 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:00:38.209 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T11:00:38.209 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:00:38.229 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1771 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:00:38.229 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:00:38.229 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T11:00:38.229 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:00:38.265 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=499 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:00:38.265 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:00:38.265 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T11:00:38.265 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:00:38.287 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1771 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:00:38.287 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:00:38.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.412+0000 7f5feb3f3640 1 Processor -- start 2024-04-25T11:00:38.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.412+0000 7f5feb3f3640 1 -- start start 2024-04-25T11:00:38.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.413+0000 7f5feb3f3640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5fe40b7750 0x7f5fe40d1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:38.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.413+0000 7f5feb3f3640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5fe40cf610 0x7f5fe4164800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:38.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.413+0000 7f5feb3f3640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fe40b3af0 con 0x7f5fe4164d40 2024-04-25T11:00:38.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.413+0000 7f5feb3f3640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f5fe4164320 con 0x7f5fe40cf610 2024-04-25T11:00:38.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.413+0000 7f5feb3f3640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5fe40b2f70 con 0x7f5fe40b7750 2024-04-25T11:00:38.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.414+0000 7f5fe9990640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f5fe4164d40 0x7f5fe406d7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:39450/0 (socket says 172.21.15.69:39450) 2024-04-25T11:00:38.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.414+0000 7f5fe9990640 1 -- 172.21.15.69:0/1934592855 learned_addr learned my addr 172.21.15.69:0/1934592855 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:38.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.414+0000 7f5fe918f640 1 --2- 172.21.15.69:0/1934592855 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5fe40b7750 0x7f5fe40d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:38.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.414+0000 7f5fe898e640 1 --2- 172.21.15.69:0/1934592855 >> v2:172.21.15.146:3300/0 conn(0x7f5fe40cf610 0x7f5fe4164800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:38.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.414+0000 7f5fe3fff640 1 -- 172.21.15.69:0/1934592855 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4237920823 0 0) 0x7f5fe40b3af0 con 0x7f5fe4164d40 2024-04-25T11:00:38.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.414+0000 7f5fe3fff640 1 -- 172.21.15.69:0/1934592855 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5fcc003520 con 0x7f5fe4164d40 2024-04-25T11:00:38.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.414+0000 7f5fe3fff640 1 -- 172.21.15.69:0/1934592855 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3540948482 0 0) 0x7f5fcc003520 con 0x7f5fe4164d40 2024-04-25T11:00:38.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.414+0000 7f5fe3fff640 1 -- 172.21.15.69:0/1934592855 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5fe40b7750 msgr2=0x7f5fe40d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:38.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.414+0000 7f5fe3fff640 1 --2- 172.21.15.69:0/1934592855 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5fe40b7750 0x7f5fe40d1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:38.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.414+0000 7f5fe3fff640 1 -- 172.21.15.69:0/1934592855 >> v2:172.21.15.146:3300/0 conn(0x7f5fe40cf610 msgr2=0x7f5fe4164800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:38.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.414+0000 7f5fe3fff640 1 --2- 172.21.15.69:0/1934592855 >> v2:172.21.15.146:3300/0 conn(0x7f5fe40cf610 0x7f5fe4164800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:38.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.414+0000 7f5fe3fff640 1 -- 172.21.15.69:0/1934592855 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe40b3250 con 0x7f5fe4164d40 2024-04-25T11:00:38.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.414+0000 7f5fe3fff640 1 -- 172.21.15.69:0/1934592855 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7f5fd8001670 con 0x7f5fe4164d40 2024-04-25T11:00:38.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.414+0000 7f5fe3fff640 1 -- 172.21.15.69:0/1934592855 >> v1:172.21.15.69:6789/0 conn(0x7f5fe4164d40 legacy=0x7f5fe406d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:38.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.414+0000 7f5fe3fff640 1 --2- 172.21.15.69:0/1934592855 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5fe40b7750 0x7f5fcc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:38.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.414+0000 7f5fe3fff640 1 --2- 172.21.15.69:0/1934592855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5fcc003f30 0x7f5fcc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:38.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.414+0000 7f5fe3fff640 1 --2- 172.21.15.69:0/1934592855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fcc004d40 0x7f5fcc005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:38.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.414+0000 7f5fe3fff640 1 -- 172.21.15.69:0/1934592855 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe4164320 con 0x7f5fcc004d40 2024-04-25T11:00:38.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.414+0000 7f5fe3fff640 1 -- 172.21.15.69:0/1934592855 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe40b2f70 con 0x7f5fcc003f30 2024-04-25T11:00:38.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.414+0000 7f5fe3fff640 1 -- 172.21.15.69:0/1934592855 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5fcc00bac0 con 0x7f5fe40b7750 2024-04-25T11:00:38.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.414+0000 7f5fe9990640 1 --2- 172.21.15.69:0/1934592855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fcc004d40 0x7f5fcc005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:38.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.414+0000 7f5fe918f640 1 --2- 172.21.15.69:0/1934592855 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5fe40b7750 0x7f5fcc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:38.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.414+0000 7f5fe898e640 1 --2- 172.21.15.69:0/1934592855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5fcc003f30 0x7f5fcc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:38.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.415+0000 7f5fe9990640 1 -- 172.21.15.69:0/1934592855 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5fe40b7750 msgr2=0x7f5fcc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:38.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.415+0000 7f5fe9990640 1 --2- 172.21.15.69:0/1934592855 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5fe40b7750 0x7f5fcc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:38.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.415+0000 7f5fe9990640 1 -- 172.21.15.69:0/1934592855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5fcc003f30 msgr2=0x7f5fcc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:38.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.415+0000 7f5fe9990640 1 --2- 172.21.15.69:0/1934592855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5fcc003f30 0x7f5fcc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:38.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.415+0000 7f5fe9990640 1 -- 172.21.15.69:0/1934592855 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fcc003520 con 0x7f5fcc004d40 2024-04-25T11:00:38.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.415+0000 7f5fe9990640 1 --2- 172.21.15.69:0/1934592855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fcc004d40 0x7f5fcc005150 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8007fd0 tx=0x7f5fcc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=484283936c8fd3db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:38.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.415+0000 7f5fe3fff640 1 -- 172.21.15.69:0/1934592855 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8007d50 con 0x7f5fcc004d40 2024-04-25T11:00:38.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.416+0000 7f5fe3fff640 1 -- 172.21.15.69:0/1934592855 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fd8005af0 con 0x7f5fcc004d40 2024-04-25T11:00:38.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.416+0000 7f5fe3fff640 1 -- 172.21.15.69:0/1934592855 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8007070 con 0x7f5fcc004d40 2024-04-25T11:00:38.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.416+0000 7f5feb3f3640 1 -- 172.21.15.69:0/1934592855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fcc004d40 msgr2=0x7f5fcc005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:38.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.416+0000 7f5feb3f3640 1 --2- 172.21.15.69:0/1934592855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fcc004d40 0x7f5fcc005150 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8007fd0 tx=0x7f5fcc005830 comp rx=0 tx=0).stop 2024-04-25T11:00:38.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.416+0000 7f5fe9990640 1 -- 172.21.15.69:0/1934592855 reap_dead start 2024-04-25T11:00:38.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.416+0000 7f5feb3f3640 1 -- 172.21.15.69:0/1934592855 shutdown_connections 2024-04-25T11:00:38.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.416+0000 7f5feb3f3640 1 -- 172.21.15.69:0/1934592855 >> 172.21.15.69:0/1934592855 conn(0x7f5fe409b7d0 msgr2=0x7f5fe40b7040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:38.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.416+0000 7f5feb3f3640 1 -- 172.21.15.69:0/1934592855 shutdown_connections 2024-04-25T11:00:38.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.416+0000 7f5feb3f3640 1 -- 172.21.15.69:0/1934592855 wait complete. 2024-04-25T11:00:38.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.418+0000 7f5feb3f3640 1 Processor -- start 2024-04-25T11:00:38.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.418+0000 7f5feb3f3640 1 -- start start 2024-04-25T11:00:38.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.420+0000 7f5feb3f3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fe41fab40 0x7f5fe4153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:38.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.420+0000 7f5feb3f3640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5fe4162e30 0x7f5fe4163240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:38.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.420+0000 7f5feb3f3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5fe41e9e60 0x7f5fe41ea270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:38.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.420+0000 7f5feb3f3640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe40cfa20 con 0x7f5fe41fab40 2024-04-25T11:00:38.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.420+0000 7f5feb3f3640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe40b3810 con 0x7f5fe41e9e60 2024-04-25T11:00:38.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.420+0000 7f5feb3f3640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5fe406def0 con 0x7f5fe4162e30 2024-04-25T11:00:38.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.420+0000 7f5fe918f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fe41fab40 0x7f5fe4153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:38.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.420+0000 7f5fe918f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fe41fab40 0x7f5fe4153dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47860/0 (socket says 172.21.15.69:47860) 2024-04-25T11:00:38.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.420+0000 7f5fe918f640 1 -- 172.21.15.69:0/2180193382 learned_addr learned my addr 172.21.15.69:0/2180193382 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:38.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.420+0000 7f5fe898e640 1 --2- 172.21.15.69:0/2180193382 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5fe4162e30 0x7f5fe4163240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:38.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.420+0000 7f5fe9990640 1 --2- 172.21.15.69:0/2180193382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5fe41e9e60 0x7f5fe41ea270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:38.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.420+0000 7f5fe918f640 1 -- 172.21.15.69:0/2180193382 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5fe4162e30 msgr2=0x7f5fe4163240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:38.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.420+0000 7f5fe918f640 1 --2- 172.21.15.69:0/2180193382 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5fe4162e30 0x7f5fe4163240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:38.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.420+0000 7f5fe918f640 1 -- 172.21.15.69:0/2180193382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5fe41e9e60 msgr2=0x7f5fe41ea270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:38.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.420+0000 7f5fe918f640 1 --2- 172.21.15.69:0/2180193382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5fe41e9e60 0x7f5fe41ea270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:38.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.420+0000 7f5fe918f640 1 -- 172.21.15.69:0/2180193382 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe4163780 con 0x7f5fe41fab40 2024-04-25T11:00:38.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.420+0000 7f5fe918f640 1 --2- 172.21.15.69:0/2180193382 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fe41fab40 0x7f5fe4153dc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc009f80 tx=0x7f5fdc00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:38.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.421+0000 7f5fe1ffb640 1 -- 172.21.15.69:0/2180193382 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc00fea0 con 0x7f5fe41fab40 2024-04-25T11:00:38.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.421+0000 7f5fe1ffb640 1 -- 172.21.15.69:0/2180193382 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fdc015630 con 0x7f5fe41fab40 2024-04-25T11:00:38.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.421+0000 7f5feb3f3640 1 -- 172.21.15.69:0/2180193382 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe4154300 con 0x7f5fe41fab40 2024-04-25T11:00:38.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.421+0000 7f5fe1ffb640 1 -- 172.21.15.69:0/2180193382 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc0159e0 con 0x7f5fe41fab40 2024-04-25T11:00:38.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.421+0000 7f5feb3f3640 1 -- 172.21.15.69:0/2180193382 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe415e090 con 0x7f5fe41fab40 2024-04-25T11:00:38.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.422+0000 7f5feb3f3640 1 -- 172.21.15.69:0/2180193382 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fe4050740 con 0x7f5fe41fab40 2024-04-25T11:00:38.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.423+0000 7f5fe1ffb640 1 -- 172.21.15.69:0/2180193382 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5fdc015b80 con 0x7f5fe41fab40 2024-04-25T11:00:38.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.423+0000 7f5fe1ffb640 1 --2- 172.21.15.69:0/2180193382 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5fd00744b0 0x7f5fd00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:38.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.423+0000 7f5fe1ffb640 1 -- 172.21.15.69:0/2180193382 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7f5fdc096160 con 0x7f5fe41fab40 2024-04-25T11:00:38.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.423+0000 7f5fe898e640 1 --2- 172.21.15.69:0/2180193382 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5fd00744b0 0x7f5fd00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:38.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.423+0000 7f5fe898e640 1 --2- 172.21.15.69:0/2180193382 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5fd00744b0 0x7f5fd00769c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009bc0 tx=0x7f5fd4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:38.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.428+0000 7f5fe1ffb640 1 -- 172.21.15.69:0/2180193382 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5fe4163780 con 0x7f5fe41fab40 2024-04-25T11:00:38.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.641+0000 7f5feb3f3640 1 -- 172.21.15.69:0/2180193382 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5fe40d0b00 con 0x7f5fe41fab40 2024-04-25T11:00:38.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.642+0000 7f5fe1ffb640 1 -- 172.21.15.69:0/2180193382 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+5180 (secure 0 0 0) 0x7f5fdc019030 con 0x7f5fe41fab40 2024-04-25T11:00:38.642 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:00:38.642 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2223802951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":2223802951},{"type":"v1","addr":"172.21.15.146:6835","nonce":2223802951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4867,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/8243043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":8243043},{"type":"v1","addr":"172.21.15.146:6834","nonce":8243043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5250,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1639802532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1639802532},{"type":"v1","addr":"172.21.15.69:6837","nonce":1639802532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:00:06.608766+0000","modified":"2024-04-25T11:00:34.079108+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5031":{"gid":5031,"name":"c","rank":0,"incarnation":17,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6836/3766456547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3766456547},{"type":"v1","addr":"172.21.15.69:6836","nonce":3766456547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5031,"qdb_cluster":[5031]},"id":2}]} 2024-04-25T11:00:38.643 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 21 2024-04-25T11:00:38.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.644+0000 7f5feb3f3640 1 -- 172.21.15.69:0/2180193382 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5fd00744b0 msgr2=0x7f5fd00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:38.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.644+0000 7f5feb3f3640 1 --2- 172.21.15.69:0/2180193382 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5fd00744b0 0x7f5fd00769c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009bc0 tx=0x7f5fd4009190 comp rx=0 tx=0).stop 2024-04-25T11:00:38.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.644+0000 7f5feb3f3640 1 -- 172.21.15.69:0/2180193382 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fe41fab40 msgr2=0x7f5fe4153dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:38.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.644+0000 7f5feb3f3640 1 --2- 172.21.15.69:0/2180193382 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fe41fab40 0x7f5fe4153dc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc009f80 tx=0x7f5fdc00f580 comp rx=0 tx=0).stop 2024-04-25T11:00:38.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.644+0000 7f5feb3f3640 1 -- 172.21.15.69:0/2180193382 shutdown_connections 2024-04-25T11:00:38.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.644+0000 7f5feb3f3640 1 --2- 172.21.15.69:0/2180193382 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5fd00744b0 0x7f5fd00769c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:38.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.644+0000 7f5feb3f3640 1 --2- 172.21.15.69:0/2180193382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5fe41e9e60 0x7f5fe41ea270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:38.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.644+0000 7f5feb3f3640 1 --2- 172.21.15.69:0/2180193382 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5fe4162e30 0x7f5fe4163240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:38.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.644+0000 7f5feb3f3640 1 --2- 172.21.15.69:0/2180193382 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fe41fab40 0x7f5fe4153dc0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:38.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.644+0000 7f5feb3f3640 1 -- 172.21.15.69:0/2180193382 >> 172.21.15.69:0/2180193382 conn(0x7f5fe409b7d0 msgr2=0x7f5fe406d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:38.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.644+0000 7f5feb3f3640 1 -- 172.21.15.69:0/2180193382 shutdown_connections 2024-04-25T11:00:38.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.644+0000 7f5feb3f3640 1 -- 172.21.15.69:0/2180193382 wait complete. 2024-04-25T11:00:38.656 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:00:38.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.763+0000 7f5d3924a640 1 Processor -- start 2024-04-25T11:00:38.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.763+0000 7f5d3924a640 1 -- start start 2024-04-25T11:00:38.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.764+0000 7f5d3924a640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5d340d4130 0x7f5d340d1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:38.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.764+0000 7f5d3924a640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d340b3c10 con 0x7f5d3406dcf0 2024-04-25T11:00:38.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.764+0000 7f5d3924a640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d340b3ef0 con 0x7f5d3406d8e0 2024-04-25T11:00:38.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.764+0000 7f5d3924a640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f5d340b9520 con 0x7f5d340d4130 2024-04-25T11:00:38.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.765+0000 7f5d3359e640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f5d3406dcf0 0x7f5d340d22a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:39464/0 (socket says 172.21.15.69:39464) 2024-04-25T11:00:38.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.765+0000 7f5d3359e640 1 -- 172.21.15.69:0/3713285572 learned_addr learned my addr 172.21.15.69:0/3713285572 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:38.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.765+0000 7f5d3259c640 1 --2- 172.21.15.69:0/3713285572 >> v2:172.21.15.146:3300/0 conn(0x7f5d340d4130 0x7f5d340d1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:38.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.765+0000 7f5d31d9b640 1 -- 172.21.15.69:0/3713285572 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2492875913 0 0) 0x7f5d340b3c10 con 0x7f5d3406dcf0 2024-04-25T11:00:38.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.765+0000 7f5d31d9b640 1 -- 172.21.15.69:0/3713285572 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d1c003520 con 0x7f5d3406dcf0 2024-04-25T11:00:38.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.765+0000 7f5d31d9b640 1 -- 172.21.15.69:0/3713285572 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 773455345 0 0) 0x7f5d340b3ef0 con 0x7f5d3406d8e0 2024-04-25T11:00:38.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.765+0000 7f5d31d9b640 1 -- 172.21.15.69:0/3713285572 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d340b3c10 con 0x7f5d3406d8e0 2024-04-25T11:00:38.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.765+0000 7f5d3259c640 1 -- 172.21.15.69:0/3713285572 >> v1:172.21.15.146:6789/0 conn(0x7f5d3406d8e0 legacy=0x7f5d340d3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:38.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.765+0000 7f5d31d9b640 1 -- 172.21.15.69:0/3713285572 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1282445016 0 0) 0x7f5d1c003520 con 0x7f5d3406dcf0 2024-04-25T11:00:38.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.765+0000 7f5d3259c640 1 -- 172.21.15.69:0/3713285572 >> v1:172.21.15.69:6789/0 conn(0x7f5d3406dcf0 legacy=0x7f5d340d22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:38.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.765+0000 7f5d3259c640 1 -- 172.21.15.69:0/3713285572 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d340b41d0 con 0x7f5d340d4130 2024-04-25T11:00:38.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.766+0000 7f5d3259c640 1 --2- 172.21.15.69:0/3713285572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d340d4130 0x7f5d340d1d60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5d280097a0 tx=0x7f5d2802a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7ef6f4ff12f8b1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:38.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.766+0000 7f5d31d9b640 1 -- 172.21.15.69:0/3713285572 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d28037070 con 0x7f5d340d4130 2024-04-25T11:00:38.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.766+0000 7f5d31d9b640 1 -- 172.21.15.69:0/3713285572 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d2802abc0 con 0x7f5d340d4130 2024-04-25T11:00:38.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.766+0000 7f5d31d9b640 1 -- 172.21.15.69:0/3713285572 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c0019f0 con 0x7f5d340d4130 2024-04-25T11:00:38.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.767+0000 7f5d3924a640 1 -- 172.21.15.69:0/3713285572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d340d4130 msgr2=0x7f5d340d1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:38.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.767+0000 7f5d3924a640 1 --2- 172.21.15.69:0/3713285572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d340d4130 0x7f5d340d1d60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5d280097a0 tx=0x7f5d2802a130 comp rx=0 tx=0).stop 2024-04-25T11:00:38.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.767+0000 7f5d3924a640 1 -- 172.21.15.69:0/3713285572 shutdown_connections 2024-04-25T11:00:38.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.767+0000 7f5d3924a640 1 --2- 172.21.15.69:0/3713285572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d340d4130 0x7f5d340d1d60 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:38.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.767+0000 7f5d3924a640 1 -- 172.21.15.69:0/3713285572 >> 172.21.15.69:0/3713285572 conn(0x7f5d3408d110 msgr2=0x7f5d3408d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:38.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.767+0000 7f5d3924a640 1 -- 172.21.15.69:0/3713285572 shutdown_connections 2024-04-25T11:00:38.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.767+0000 7f5d3924a640 1 -- 172.21.15.69:0/3713285572 wait complete. 2024-04-25T11:00:38.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.769+0000 7f5d3924a640 1 Processor -- start 2024-04-25T11:00:38.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.769+0000 7f5d3924a640 1 -- start start 2024-04-25T11:00:38.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.770+0000 7f5d3924a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d3406d8e0 0x7f5d341eadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:38.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.770+0000 7f5d3924a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d3406dcf0 0x7f5d341f9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:38.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.770+0000 7f5d3924a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d3414e9e0 0x7f5d341f9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:38.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.770+0000 7f5d3924a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5d340b38e0 con 0x7f5d3406dcf0 2024-04-25T11:00:38.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.770+0000 7f5d3924a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5d340b4820 con 0x7f5d3414e9e0 2024-04-25T11:00:38.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.770+0000 7f5d3924a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5d340d29b0 con 0x7f5d3406d8e0 2024-04-25T11:00:38.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.770+0000 7f5d3259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d3406dcf0 0x7f5d341f9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:38.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.770+0000 7f5d3259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d3406dcf0 0x7f5d341f9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47864/0 (socket says 172.21.15.69:47864) 2024-04-25T11:00:38.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.770+0000 7f5d3259c640 1 -- 172.21.15.69:0/1009698147 learned_addr learned my addr 172.21.15.69:0/1009698147 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:38.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.770+0000 7f5d32d9d640 1 --2- 172.21.15.69:0/1009698147 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d3406d8e0 0x7f5d341eadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:38.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.770+0000 7f5d3359e640 1 --2- 172.21.15.69:0/1009698147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d3414e9e0 0x7f5d341f9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:38.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.771+0000 7f5d3259c640 1 -- 172.21.15.69:0/1009698147 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d3406d8e0 msgr2=0x7f5d341eadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:38.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.771+0000 7f5d3259c640 1 --2- 172.21.15.69:0/1009698147 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d3406d8e0 0x7f5d341eadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:38.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.771+0000 7f5d3259c640 1 -- 172.21.15.69:0/1009698147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d3414e9e0 msgr2=0x7f5d341f9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:38.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.771+0000 7f5d3259c640 1 --2- 172.21.15.69:0/1009698147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d3414e9e0 0x7f5d341f9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:38.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.771+0000 7f5d3259c640 1 -- 172.21.15.69:0/1009698147 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3409f100 con 0x7f5d3406dcf0 2024-04-25T11:00:38.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.771+0000 7f5d3259c640 1 --2- 172.21.15.69:0/1009698147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d3406dcf0 0x7f5d341f9780 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5d280098c0 tx=0x7f5d280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:38.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.771+0000 7f5d1b7fe640 1 -- 172.21.15.69:0/1009698147 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d28042020 con 0x7f5d3406dcf0 2024-04-25T11:00:38.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.771+0000 7f5d1b7fe640 1 -- 172.21.15.69:0/1009698147 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d2803d670 con 0x7f5d3406dcf0 2024-04-25T11:00:38.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.771+0000 7f5d1b7fe640 1 -- 172.21.15.69:0/1009698147 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d28037040 con 0x7f5d3406dcf0 2024-04-25T11:00:38.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.771+0000 7f5d3924a640 1 -- 172.21.15.69:0/1009698147 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d3409f600 con 0x7f5d3406dcf0 2024-04-25T11:00:38.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.771+0000 7f5d3924a640 1 -- 172.21.15.69:0/1009698147 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d3409fbc0 con 0x7f5d3406dcf0 2024-04-25T11:00:38.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.772+0000 7f5d3924a640 1 -- 172.21.15.69:0/1009698147 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d34050740 con 0x7f5d3406dcf0 2024-04-25T11:00:38.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.773+0000 7f5d1b7fe640 1 -- 172.21.15.69:0/1009698147 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5d28045020 con 0x7f5d3406dcf0 2024-04-25T11:00:38.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.773+0000 7f5d1b7fe640 1 --2- 172.21.15.69:0/1009698147 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5d200744e0 0x7f5d200769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:38.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.773+0000 7f5d1b7fe640 1 -- 172.21.15.69:0/1009698147 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7f5d280b57c0 con 0x7f5d3406dcf0 2024-04-25T11:00:38.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.773+0000 7f5d32d9d640 1 --2- 172.21.15.69:0/1009698147 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5d200744e0 0x7f5d200769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:38.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.773+0000 7f5d32d9d640 1 --2- 172.21.15.69:0/1009698147 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5d200744e0 0x7f5d200769f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5d240011e0 tx=0x7f5d24009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:38.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.777+0000 7f5d1b7fe640 1 -- 172.21.15.69:0/1009698147 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5d2804c070 con 0x7f5d3406dcf0 2024-04-25T11:00:38.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.956+0000 7f5d3924a640 1 -- 172.21.15.69:0/1009698147 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d3409f390 con 0x7f5d3406dcf0 2024-04-25T11:00:38.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.957+0000 7f5d1b7fe640 1 -- 172.21.15.69:0/1009698147 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16798 (secure 0 0 0) 0x7f5d2807fef0 con 0x7f5d3406dcf0 2024-04-25T11:00:38.958 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:00:38.958 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":32,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T11:00:30.066894+0000","last_up_change":"2024-04-25T10:59:05.546839+0000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:59:06.422674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:00:05.007593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T11:00:05.558880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6825","nonce":1876101065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6827","nonce":1876101065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6831","nonce":1876101065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6829","nonce":1876101065}]},"public_addr":"172.21.15.69:6825/1876101065","cluster_addr":"172.21.15.69:6827/1876101065","heartbeat_back_addr":"172.21.15.69:6831/1876101065","heartbeat_front_addr":"172.21.15.69:6829/1876101065","state":["exists","up"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6801","nonce":1806438563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6803","nonce":1806438563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6807","nonce":1806438563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6805","nonce":1806438563}]},"public_addr":"172.21.15.69:6801/1806438563","cluster_addr":"172.21.15.69:6803/1806438563","heartbeat_back_addr":"172.21.15.69:6807/1806438563","heartbeat_front_addr":"172.21.15.69:6805/1806438563","state":["exists","up"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6817","nonce":1593617983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6819","nonce":1593617983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6823","nonce":1593617983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6821","nonce":1593617983}]},"public_addr":"172.21.15.69:6817/1593617983","cluster_addr":"172.21.15.69:6819/1593617983","heartbeat_back_addr":"172.21.15.69:6823/1593617983","heartbeat_front_addr":"172.21.15.69:6821/1593617983","state":["exists","up"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6809","nonce":3950661294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6811","nonce":3950661294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6815","nonce":3950661294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6813","nonce":3950661294}]},"public_addr":"172.21.15.69:6809/3950661294","cluster_addr":"172.21.15.69:6811/3950661294","heartbeat_back_addr":"172.21.15.69:6815/3950661294","heartbeat_front_addr":"172.21.15.69:6813/3950661294","state":["exists","up"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6801","nonce":2508929872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6803","nonce":2508929872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6807","nonce":2508929872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6805","nonce":2508929872}]},"public_addr":"172.21.15.146:6801/2508929872","cluster_addr":"172.21.15.146:6803/2508929872","heartbeat_back_addr":"172.21.15.146:6807/2508929872","heartbeat_front_addr":"172.21.15.146:6805/2508929872","state":["exists","up"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6811","nonce":1310911138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6816","nonce":1310911138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6823","nonce":1310911138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6821","nonce":1310911138}]},"public_addr":"172.21.15.146:6811/1310911138","cluster_addr":"172.21.15.146:6816/1310911138","heartbeat_back_addr":"172.21.15.146:6823/1310911138","heartbeat_front_addr":"172.21.15.146:6821/1310911138","state":["exists","up"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6810","nonce":634688021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6813","nonce":634688021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6820","nonce":634688021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6817","nonce":634688021}]},"public_addr":"172.21.15.146:6810/634688021","cluster_addr":"172.21.15.146:6813/634688021","heartbeat_back_addr":"172.21.15.146:6820/634688021","heartbeat_front_addr":"172.21.15.146:6817/634688021","state":["exists","up"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6825","nonce":3814215935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6827","nonce":3814215935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6831","nonce":3814215935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6829","nonce":3814215935}]},"public_addr":"172.21.15.146:6825/3814215935","cluster_addr":"172.21.15.146:6827/3814215935","heartbeat_back_addr":"172.21.15.146:6831/3814215935","heartbeat_front_addr":"172.21.15.146:6829/3814215935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.820200+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.850189+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.764560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.866248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.807897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.869881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.782310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.827131+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6837/3153221129":"2024-04-26T11:00:30.066863+0000","172.21.15.69:6835/3153221129":"2024-04-26T11:00:30.066863+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:00:38.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.959+0000 7f5d3924a640 1 -- 172.21.15.69:0/1009698147 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5d200744e0 msgr2=0x7f5d200769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:38.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.959+0000 7f5d3924a640 1 --2- 172.21.15.69:0/1009698147 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5d200744e0 0x7f5d200769f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5d240011e0 tx=0x7f5d24009040 comp rx=0 tx=0).stop 2024-04-25T11:00:38.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.959+0000 7f5d3924a640 1 -- 172.21.15.69:0/1009698147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d3406dcf0 msgr2=0x7f5d341f9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:38.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.959+0000 7f5d3924a640 1 --2- 172.21.15.69:0/1009698147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d3406dcf0 0x7f5d341f9780 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5d280098c0 tx=0x7f5d280046c0 comp rx=0 tx=0).stop 2024-04-25T11:00:38.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.959+0000 7f5d3924a640 1 -- 172.21.15.69:0/1009698147 shutdown_connections 2024-04-25T11:00:38.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.959+0000 7f5d3924a640 1 --2- 172.21.15.69:0/1009698147 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5d200744e0 0x7f5d200769f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:38.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.959+0000 7f5d3924a640 1 --2- 172.21.15.69:0/1009698147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d3414e9e0 0x7f5d341f9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:38.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.959+0000 7f5d3924a640 1 --2- 172.21.15.69:0/1009698147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d3406dcf0 0x7f5d341f9780 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:38.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.959+0000 7f5d3924a640 1 --2- 172.21.15.69:0/1009698147 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d3406d8e0 0x7f5d341eadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:38.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.959+0000 7f5d3924a640 1 -- 172.21.15.69:0/1009698147 >> 172.21.15.69:0/1009698147 conn(0x7f5d3408d110 msgr2=0x7f5d3406e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:38.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.959+0000 7f5d3924a640 1 -- 172.21.15.69:0/1009698147 shutdown_connections 2024-04-25T11:00:38.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:38.959+0000 7f5d3924a640 1 -- 172.21.15.69:0/1009698147 wait complete. 2024-04-25T11:00:38.971 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:59:06.422674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:00:05.007593+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '30', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:00:05.558880+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:00:38.972 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:00:39.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.083+0000 7f7de9e42640 1 Processor -- start 2024-04-25T11:00:39.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.083+0000 7f7de9e42640 1 -- start start 2024-04-25T11:00:39.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.084+0000 7f7de9e42640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7de406ccd0 0x7f7de40d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:39.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.084+0000 7f7de9e42640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7de40d4a50 0x7f7de40d4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:39.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.084+0000 7f7de9e42640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7de40d1850 con 0x7f7de40d0e00 2024-04-25T11:00:39.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.084+0000 7f7de9e42640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f7de4160de0 con 0x7f7de406ccd0 2024-04-25T11:00:39.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.084+0000 7f7de9e42640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f7de41610c0 con 0x7f7de40d4a50 2024-04-25T11:00:39.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.084+0000 7f7ddbfff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7de406ccd0 0x7f7de40d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:39.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.084+0000 7f7ddbfff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f7de406ccd0 0x7f7de40d1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47872/0 (socket says 172.21.15.69:47872) 2024-04-25T11:00:39.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.084+0000 7f7ddbfff640 1 -- 172.21.15.69:0/3494498423 learned_addr learned my addr 172.21.15.69:0/3494498423 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:39.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.084+0000 7f7de3fff640 1 --2- 172.21.15.69:0/3494498423 >> v2:172.21.15.146:3300/0 conn(0x7f7de40d4a50 0x7f7de40d4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:39.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.084+0000 7f7ddbfff640 1 -- 172.21.15.69:0/3494498423 >> v2:172.21.15.146:3300/0 conn(0x7f7de40d4a50 msgr2=0x7f7de40d4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:39.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.084+0000 7f7ddbfff640 1 --2- 172.21.15.69:0/3494498423 >> v2:172.21.15.146:3300/0 conn(0x7f7de40d4a50 0x7f7de40d4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:39.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.084+0000 7f7ddbfff640 1 -- 172.21.15.69:0/3494498423 >> v1:172.21.15.146:6789/0 conn(0x7f7de40d0e00 legacy=0x7f7de406c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:39.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.084+0000 7f7ddbfff640 1 -- 172.21.15.69:0/3494498423 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de41613a0 con 0x7f7de406ccd0 2024-04-25T11:00:39.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.085+0000 7f7ddbfff640 1 --2- 172.21.15.69:0/3494498423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7de406ccd0 0x7f7de40d1210 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7de40d1750 tx=0x7f7dcc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76763720bc65e33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:39.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.085+0000 7f7de2ffd640 1 -- 172.21.15.69:0/3494498423 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc037070 con 0x7f7de406ccd0 2024-04-25T11:00:39.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.085+0000 7f7de2ffd640 1 -- 172.21.15.69:0/3494498423 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dcc032440 con 0x7f7de406ccd0 2024-04-25T11:00:39.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.085+0000 7f7de2ffd640 1 -- 172.21.15.69:0/3494498423 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc032810 con 0x7f7de406ccd0 2024-04-25T11:00:39.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.086+0000 7f7de9e42640 1 -- 172.21.15.69:0/3494498423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7de406ccd0 msgr2=0x7f7de40d1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:39.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.086+0000 7f7de9e42640 1 --2- 172.21.15.69:0/3494498423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7de406ccd0 0x7f7de40d1210 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7de40d1750 tx=0x7f7dcc02a1a0 comp rx=0 tx=0).stop 2024-04-25T11:00:39.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.086+0000 7f7de9e42640 1 -- 172.21.15.69:0/3494498423 shutdown_connections 2024-04-25T11:00:39.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.086+0000 7f7de9e42640 1 --2- 172.21.15.69:0/3494498423 >> v2:172.21.15.146:3300/0 conn(0x7f7de40d4a50 0x7f7de40d4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:39.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.086+0000 7f7de9e42640 1 --2- 172.21.15.69:0/3494498423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7de406ccd0 0x7f7de40d1210 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:39.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.086+0000 7f7de9e42640 1 -- 172.21.15.69:0/3494498423 >> 172.21.15.69:0/3494498423 conn(0x7f7de40d5d90 msgr2=0x7f7de40d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:39.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.086+0000 7f7de9e42640 1 -- 172.21.15.69:0/3494498423 shutdown_connections 2024-04-25T11:00:39.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.086+0000 7f7de9e42640 1 -- 172.21.15.69:0/3494498423 wait complete. 2024-04-25T11:00:39.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.088+0000 7f7de9e42640 1 Processor -- start 2024-04-25T11:00:39.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.088+0000 7f7de9e42640 1 -- start start 2024-04-25T11:00:39.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.089+0000 7f7de9e42640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7de40d0e00 0x7f7de41ea6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:39.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.089+0000 7f7de9e42640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7de40d4a50 0x7f7de40b5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:39.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.089+0000 7f7de9e42640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7de4157e90 0x7f7de41575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:39.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.089+0000 7f7de9e42640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7de4164e00 con 0x7f7de4157e90 2024-04-25T11:00:39.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.089+0000 7f7de9e42640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7de40b5320 con 0x7f7de40d0e00 2024-04-25T11:00:39.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.089+0000 7f7de9e42640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f7de4164f80 con 0x7f7de40d4a50 2024-04-25T11:00:39.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.089+0000 7f7de3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7de4157e90 0x7f7de41575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:39.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.089+0000 7f7de3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7de4157e90 0x7f7de41575d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47880/0 (socket says 172.21.15.69:47880) 2024-04-25T11:00:39.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.089+0000 7f7de3fff640 1 -- 172.21.15.69:0/1714828452 learned_addr learned my addr 172.21.15.69:0/1714828452 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:39.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.090+0000 7f7ddbfff640 1 --2- 172.21.15.69:0/1714828452 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7de40d4a50 0x7f7de40b5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:39.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.090+0000 7f7de37fe640 1 --2- 172.21.15.69:0/1714828452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7de40d0e00 0x7f7de41ea6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:39.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.090+0000 7f7de3fff640 1 -- 172.21.15.69:0/1714828452 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7de40d4a50 msgr2=0x7f7de40b5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:39.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.090+0000 7f7de3fff640 1 --2- 172.21.15.69:0/1714828452 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7de40d4a50 0x7f7de40b5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:39.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.090+0000 7f7de3fff640 1 -- 172.21.15.69:0/1714828452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7de40d0e00 msgr2=0x7f7de41ea6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:39.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.090+0000 7f7de3fff640 1 --2- 172.21.15.69:0/1714828452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7de40d0e00 0x7f7de41ea6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:39.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.090+0000 7f7de3fff640 1 -- 172.21.15.69:0/1714828452 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de41ec3e0 con 0x7f7de4157e90 2024-04-25T11:00:39.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.090+0000 7f7de37fe640 1 --2- 172.21.15.69:0/1714828452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7de40d0e00 0x7f7de41ea6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:00:39.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.090+0000 7f7de3fff640 1 --2- 172.21.15.69:0/1714828452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7de4157e90 0x7f7de41575d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc007e90 tx=0x7f7ddc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:39.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.090+0000 7f7de0ff9640 1 -- 172.21.15.69:0/1714828452 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc014420 con 0x7f7de4157e90 2024-04-25T11:00:39.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.090+0000 7f7de0ff9640 1 -- 172.21.15.69:0/1714828452 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ddc019070 con 0x7f7de4157e90 2024-04-25T11:00:39.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.090+0000 7f7de9e42640 1 -- 172.21.15.69:0/1714828452 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7de41ec6c0 con 0x7f7de4157e90 2024-04-25T11:00:39.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.090+0000 7f7de0ff9640 1 -- 172.21.15.69:0/1714828452 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc014800 con 0x7f7de4157e90 2024-04-25T11:00:39.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.090+0000 7f7de9e42640 1 -- 172.21.15.69:0/1714828452 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7de41ea190 con 0x7f7de4157e90 2024-04-25T11:00:39.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.091+0000 7f7de9e42640 1 -- 172.21.15.69:0/1714828452 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7de40d1e80 con 0x7f7de4157e90 2024-04-25T11:00:39.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.092+0000 7f7de0ff9640 1 -- 172.21.15.69:0/1714828452 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7ddc0149a0 con 0x7f7de4157e90 2024-04-25T11:00:39.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.092+0000 7f7de0ff9640 1 --2- 172.21.15.69:0/1714828452 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f7dc40744e0 0x7f7dc40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:39.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.092+0000 7f7de0ff9640 1 -- 172.21.15.69:0/1714828452 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7f7ddc095af0 con 0x7f7de4157e90 2024-04-25T11:00:39.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.092+0000 7f7de37fe640 1 --2- 172.21.15.69:0/1714828452 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f7dc40744e0 0x7f7dc40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:39.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.093+0000 7f7de37fe640 1 --2- 172.21.15.69:0/1714828452 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f7dc40744e0 0x7f7dc40769f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7dd400a830 tx=0x7f7dd4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:39.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.097+0000 7f7de0ff9640 1 -- 172.21.15.69:0/1714828452 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7de40d1e80 con 0x7f7de4157e90 2024-04-25T11:00:39.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.278+0000 7f7de9e42640 1 -- 172.21.15.69:0/1714828452 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7de4050740 con 0x7f7de4157e90 2024-04-25T11:00:39.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.279+0000 7f7de0ff9640 1 -- 172.21.15.69:0/1714828452 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) ==== 93+0+10 (secure 0 0 0) 0x7f7ddc05fbd0 con 0x7f7de4157e90 2024-04-25T11:00:39.280 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-25T11:00:39.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.281+0000 7f7de9e42640 1 -- 172.21.15.69:0/1714828452 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f7dc40744e0 msgr2=0x7f7dc40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:39.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.281+0000 7f7de9e42640 1 --2- 172.21.15.69:0/1714828452 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f7dc40744e0 0x7f7dc40769f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7dd400a830 tx=0x7f7dd4008040 comp rx=0 tx=0).stop 2024-04-25T11:00:39.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.281+0000 7f7de9e42640 1 -- 172.21.15.69:0/1714828452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7de4157e90 msgr2=0x7f7de41575d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:39.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.281+0000 7f7de9e42640 1 --2- 172.21.15.69:0/1714828452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7de4157e90 0x7f7de41575d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc007e90 tx=0x7f7ddc00e4b0 comp rx=0 tx=0).stop 2024-04-25T11:00:39.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.281+0000 7f7de9e42640 1 -- 172.21.15.69:0/1714828452 shutdown_connections 2024-04-25T11:00:39.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.281+0000 7f7de9e42640 1 --2- 172.21.15.69:0/1714828452 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f7dc40744e0 0x7f7dc40769f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:39.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.281+0000 7f7de9e42640 1 --2- 172.21.15.69:0/1714828452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7de4157e90 0x7f7de41575d0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:39.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.281+0000 7f7de9e42640 1 --2- 172.21.15.69:0/1714828452 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7de40d4a50 0x7f7de40b5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:39.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.281+0000 7f7de9e42640 1 --2- 172.21.15.69:0/1714828452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7de40d0e00 0x7f7de41ea6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:39.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.281+0000 7f7de9e42640 1 -- 172.21.15.69:0/1714828452 >> 172.21.15.69:0/1714828452 conn(0x7f7de40d5d90 msgr2=0x7f7de41618a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:39.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.281+0000 7f7de9e42640 1 -- 172.21.15.69:0/1714828452 shutdown_connections 2024-04-25T11:00:39.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.281+0000 7f7de9e42640 1 -- 172.21.15.69:0/1714828452 wait complete. 2024-04-25T11:00:39.293 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:00:39.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.402+0000 7f69a51eb640 1 Processor -- start 2024-04-25T11:00:39.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.402+0000 7f69a51eb640 1 -- start start 2024-04-25T11:00:39.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.403+0000 7f69a51eb640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f69a006dce0 0x7f69a00704c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:39.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.403+0000 7f69a51eb640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69a006fa20 con 0x7f69a00b0c90 2024-04-25T11:00:39.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.403+0000 7f69a51eb640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69a0070fd0 con 0x7f69a015adf0 2024-04-25T11:00:39.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.403+0000 7f69a51eb640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f69a00712b0 con 0x7f69a006dce0 2024-04-25T11:00:39.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.403+0000 7f699ed9d640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f69a00b0c90 0x7f69a006d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:39466/0 (socket says 172.21.15.69:39466) 2024-04-25T11:00:39.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.403+0000 7f699ed9d640 1 -- 172.21.15.69:0/4112728369 learned_addr learned my addr 172.21.15.69:0/4112728369 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:39.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.404+0000 7f6997fff640 1 --2- 172.21.15.69:0/4112728369 >> v2:172.21.15.146:3300/0 conn(0x7f69a006dce0 0x7f69a00704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:39.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.404+0000 7f699e59c640 1 -- 172.21.15.69:0/4112728369 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3925918256 0 0) 0x7f69a006fa20 con 0x7f69a00b0c90 2024-04-25T11:00:39.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.404+0000 7f699e59c640 1 -- 172.21.15.69:0/4112728369 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f698c005730 con 0x7f69a00b0c90 2024-04-25T11:00:39.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.404+0000 7f699e59c640 1 -- 172.21.15.69:0/4112728369 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4246325353 0 0) 0x7f698c005730 con 0x7f69a00b0c90 2024-04-25T11:00:39.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.404+0000 7f699e59c640 1 -- 172.21.15.69:0/4112728369 >> v2:172.21.15.146:3300/0 conn(0x7f69a006dce0 msgr2=0x7f69a00704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:39.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.404+0000 7f699e59c640 1 --2- 172.21.15.69:0/4112728369 >> v2:172.21.15.146:3300/0 conn(0x7f69a006dce0 0x7f69a00704c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:39.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.404+0000 7f699e59c640 1 -- 172.21.15.69:0/4112728369 >> v1:172.21.15.146:6789/0 conn(0x7f69a015adf0 legacy=0x7f69a015d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:39.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.404+0000 7f699e59c640 1 -- 172.21.15.69:0/4112728369 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a006fcc0 con 0x7f69a00b0c90 2024-04-25T11:00:39.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.404+0000 7f699e59c640 1 -- 172.21.15.69:0/4112728369 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7f6990001690 con 0x7f69a00b0c90 2024-04-25T11:00:39.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.404+0000 7f699e59c640 1 -- 172.21.15.69:0/4112728369 >> v1:172.21.15.69:6789/0 conn(0x7f69a00b0c90 legacy=0x7f69a006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:39.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.404+0000 7f699e59c640 1 --2- 172.21.15.69:0/4112728369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f698c005bc0 0x7f698c005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:39.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.404+0000 7f699e59c640 1 --2- 172.21.15.69:0/4112728369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f698c006a10 0x7f698c006e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:39.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.404+0000 7f699e59c640 1 --2- 172.21.15.69:0/4112728369 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f698c007360 0x7f698c00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:39.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.404+0000 7f699e59c640 1 -- 172.21.15.69:0/4112728369 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f69a00712b0 con 0x7f698c005bc0 2024-04-25T11:00:39.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.404+0000 7f699e59c640 1 -- 172.21.15.69:0/4112728369 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f698c00e2d0 con 0x7f698c006a10 2024-04-25T11:00:39.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.404+0000 7f699e59c640 1 -- 172.21.15.69:0/4112728369 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f698c00e4d0 con 0x7f698c007360 2024-04-25T11:00:39.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.404+0000 7f699ed9d640 1 --2- 172.21.15.69:0/4112728369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f698c005bc0 0x7f698c005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:39.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.404+0000 7f6997fff640 1 --2- 172.21.15.69:0/4112728369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f698c006a10 0x7f698c006e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:39.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.404+0000 7f699f59e640 1 --2- 172.21.15.69:0/4112728369 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f698c007360 0x7f698c00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:39.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.405+0000 7f699ed9d640 1 -- 172.21.15.69:0/4112728369 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f698c007360 msgr2=0x7f698c00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:39.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.405+0000 7f699ed9d640 1 --2- 172.21.15.69:0/4112728369 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f698c007360 0x7f698c00dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:39.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.405+0000 7f699ed9d640 1 -- 172.21.15.69:0/4112728369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f698c006a10 msgr2=0x7f698c006e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:39.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.405+0000 7f699ed9d640 1 --2- 172.21.15.69:0/4112728369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f698c006a10 0x7f698c006e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:39.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.405+0000 7f699f59e640 1 -- 172.21.15.69:0/4112728369 reap_dead start 2024-04-25T11:00:39.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.405+0000 7f699ed9d640 1 -- 172.21.15.69:0/4112728369 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698c005730 con 0x7f698c005bc0 2024-04-25T11:00:39.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.405+0000 7f699ed9d640 1 --2- 172.21.15.69:0/4112728369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f698c005bc0 0x7f698c005ff0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f69a00b6c80 tx=0x7f698c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6b854f3c4d9ce1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:39.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.406+0000 7f699e59c640 1 -- 172.21.15.69:0/4112728369 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6990007d30 con 0x7f698c005bc0 2024-04-25T11:00:39.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.406+0000 7f699e59c640 1 -- 172.21.15.69:0/4112728369 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6990005ad0 con 0x7f698c005bc0 2024-04-25T11:00:39.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.406+0000 7f699e59c640 1 -- 172.21.15.69:0/4112728369 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6990007070 con 0x7f698c005bc0 2024-04-25T11:00:39.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.406+0000 7f69a51eb640 1 -- 172.21.15.69:0/4112728369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f698c005bc0 msgr2=0x7f698c005ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:39.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.406+0000 7f69a51eb640 1 --2- 172.21.15.69:0/4112728369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f698c005bc0 0x7f698c005ff0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f69a00b6c80 tx=0x7f698c007a40 comp rx=0 tx=0).stop 2024-04-25T11:00:39.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.406+0000 7f69a51eb640 1 -- 172.21.15.69:0/4112728369 shutdown_connections 2024-04-25T11:00:39.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.406+0000 7f69a51eb640 1 --2- 172.21.15.69:0/4112728369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f698c005bc0 0x7f698c005ff0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:39.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.406+0000 7f69a51eb640 1 -- 172.21.15.69:0/4112728369 >> 172.21.15.69:0/4112728369 conn(0x7f69a0073090 msgr2=0x7f69a00734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:39.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.406+0000 7f69a51eb640 1 -- 172.21.15.69:0/4112728369 shutdown_connections 2024-04-25T11:00:39.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.406+0000 7f69a51eb640 1 -- 172.21.15.69:0/4112728369 wait complete. 2024-04-25T11:00:39.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.408+0000 7f69a51eb640 1 Processor -- start 2024-04-25T11:00:39.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.408+0000 7f69a51eb640 1 -- start start 2024-04-25T11:00:39.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.409+0000 7f69a51eb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f698c005bc0 0x7f69a015cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:39.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.409+0000 7f69a51eb640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69a0148070 0x7f69a00b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:39.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.409+0000 7f69a51eb640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69a00b5240 0x7f69a0148800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:39.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.409+0000 7f69a51eb640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f69a00704c0 con 0x7f698c005bc0 2024-04-25T11:00:39.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.410+0000 7f69a51eb640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f69a0070c80 con 0x7f69a00b5240 2024-04-25T11:00:39.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.410+0000 7f69a51eb640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f69a0070640 con 0x7f69a0148070 2024-04-25T11:00:39.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.410+0000 7f699ed9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f698c005bc0 0x7f69a015cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:39.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.410+0000 7f699ed9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f698c005bc0 0x7f69a015cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47904/0 (socket says 172.21.15.69:47904) 2024-04-25T11:00:39.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.410+0000 7f699ed9d640 1 -- 172.21.15.69:0/2725614918 learned_addr learned my addr 172.21.15.69:0/2725614918 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:39.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.410+0000 7f6997fff640 1 --2- 172.21.15.69:0/2725614918 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69a0148070 0x7f69a00b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:39.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.410+0000 7f699f59e640 1 --2- 172.21.15.69:0/2725614918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69a00b5240 0x7f69a0148800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:39.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.410+0000 7f699ed9d640 1 -- 172.21.15.69:0/2725614918 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69a0148070 msgr2=0x7f69a00b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:39.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.410+0000 7f699ed9d640 1 --2- 172.21.15.69:0/2725614918 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69a0148070 0x7f69a00b4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:39.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.410+0000 7f699ed9d640 1 -- 172.21.15.69:0/2725614918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69a00b5240 msgr2=0x7f69a0148800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:39.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.410+0000 7f699ed9d640 1 --2- 172.21.15.69:0/2725614918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69a00b5240 0x7f69a0148800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:39.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.410+0000 7f699ed9d640 1 -- 172.21.15.69:0/2725614918 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a0156f50 con 0x7f698c005bc0 2024-04-25T11:00:39.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.410+0000 7f699ed9d640 1 --2- 172.21.15.69:0/2725614918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f698c005bc0 0x7f69a015cc20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f699002ad80 tx=0x7f69a006fe90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:39.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.410+0000 7f69977fe640 1 -- 172.21.15.69:0/2725614918 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6990037040 con 0x7f698c005bc0 2024-04-25T11:00:39.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.410+0000 7f69977fe640 1 -- 172.21.15.69:0/2725614918 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6990041070 con 0x7f698c005bc0 2024-04-25T11:00:39.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.410+0000 7f69977fe640 1 -- 172.21.15.69:0/2725614918 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6990007d30 con 0x7f698c005bc0 2024-04-25T11:00:39.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.410+0000 7f69a51eb640 1 -- 172.21.15.69:0/2725614918 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69a00b5650 con 0x7f698c005bc0 2024-04-25T11:00:39.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.410+0000 7f69a51eb640 1 -- 172.21.15.69:0/2725614918 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69a0152aa0 con 0x7f698c005bc0 2024-04-25T11:00:39.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.411+0000 7f69a51eb640 1 -- 172.21.15.69:0/2725614918 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69a0050740 con 0x7f698c005bc0 2024-04-25T11:00:39.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.412+0000 7f69977fe640 1 -- 172.21.15.69:0/2725614918 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f69900049e0 con 0x7f698c005bc0 2024-04-25T11:00:39.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.412+0000 7f69977fe640 1 --2- 172.21.15.69:0/2725614918 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f69800744b0 0x7f69800769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:39.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.413+0000 7f69977fe640 1 -- 172.21.15.69:0/2725614918 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7f69900b49e0 con 0x7f698c005bc0 2024-04-25T11:00:39.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.413+0000 7f6997fff640 1 --2- 172.21.15.69:0/2725614918 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f69800744b0 0x7f69800769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:39.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.413+0000 7f6997fff640 1 --2- 172.21.15.69:0/2725614918 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f69800744b0 0x7f69800769c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f69a00b59e0 tx=0x7f6988009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:39.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.417+0000 7f69977fe640 1 -- 172.21.15.69:0/2725614918 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f69a00b5650 con 0x7f698c005bc0 2024-04-25T11:00:39.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.589+0000 7f69a51eb640 1 -- 172.21.15.69:0/2725614918 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f69a0073820 con 0x7f698c005bc0 2024-04-25T11:00:39.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.590+0000 7f69977fe640 1 -- 172.21.15.69:0/2725614918 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v32) ==== 104+0+11 (secure 0 0 0) 0x7f699007f1a0 con 0x7f698c005bc0 2024-04-25T11:00:39.591 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-25T11:00:39.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.592+0000 7f69a51eb640 1 -- 172.21.15.69:0/2725614918 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f69800744b0 msgr2=0x7f69800769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:39.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.592+0000 7f69a51eb640 1 --2- 172.21.15.69:0/2725614918 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f69800744b0 0x7f69800769c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f69a00b59e0 tx=0x7f6988009190 comp rx=0 tx=0).stop 2024-04-25T11:00:39.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.592+0000 7f69a51eb640 1 -- 172.21.15.69:0/2725614918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f698c005bc0 msgr2=0x7f69a015cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:39.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.592+0000 7f69a51eb640 1 --2- 172.21.15.69:0/2725614918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f698c005bc0 0x7f69a015cc20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f699002ad80 tx=0x7f69a006fe90 comp rx=0 tx=0).stop 2024-04-25T11:00:39.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.592+0000 7f69a51eb640 1 -- 172.21.15.69:0/2725614918 shutdown_connections 2024-04-25T11:00:39.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.592+0000 7f69a51eb640 1 --2- 172.21.15.69:0/2725614918 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f69800744b0 0x7f69800769c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:39.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.592+0000 7f69a51eb640 1 --2- 172.21.15.69:0/2725614918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69a00b5240 0x7f69a0148800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:39.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.592+0000 7f69a51eb640 1 --2- 172.21.15.69:0/2725614918 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f69a0148070 0x7f69a00b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:39.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.592+0000 7f69a51eb640 1 --2- 172.21.15.69:0/2725614918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f698c005bc0 0x7f69a015cc20 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:39.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.592+0000 7f69a51eb640 1 -- 172.21.15.69:0/2725614918 >> 172.21.15.69:0/2725614918 conn(0x7f69a0073090 msgr2=0x7f69a0150c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:39.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.592+0000 7f69a51eb640 1 -- 172.21.15.69:0/2725614918 shutdown_connections 2024-04-25T11:00:39.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.592+0000 7f69a51eb640 1 -- 172.21.15.69:0/2725614918 wait complete. 2024-04-25T11:00:39.603 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:00:39.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.712+0000 7efce3a8c640 1 Processor -- start 2024-04-25T11:00:39.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.712+0000 7efce3a8c640 1 -- start start 2024-04-25T11:00:39.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.713+0000 7efce3a8c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7efcdc0d3000 0x7efcdc0cef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:39.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.713+0000 7efce3a8c640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcdc0cfe60 con 0x7efcdc06c5c0 2024-04-25T11:00:39.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.713+0000 7efce3a8c640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcdc0d0140 con 0x7efcdc15e3b0 2024-04-25T11:00:39.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.713+0000 7efce3a8c640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7efcdc0b3bf0 con 0x7efcdc0d3000 2024-04-25T11:00:39.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.713+0000 7efce1828640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7efcdc06c5c0 0x7efcdc0cf6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:39472/0 (socket says 172.21.15.69:39472) 2024-04-25T11:00:39.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.713+0000 7efce1828640 1 -- 172.21.15.69:0/1302246735 learned_addr learned my addr 172.21.15.69:0/1302246735 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:39.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.714+0000 7efce0826640 1 -- 172.21.15.69:0/1302246735 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3358950229 0 0) 0x7efcdc0cfe60 con 0x7efcdc06c5c0 2024-04-25T11:00:39.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.714+0000 7efce2029640 1 --2- 172.21.15.69:0/1302246735 >> v2:172.21.15.146:3300/0 conn(0x7efcdc0d3000 0x7efcdc0cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:39.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.714+0000 7efce0826640 1 -- 172.21.15.69:0/1302246735 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efcc8003520 con 0x7efcdc06c5c0 2024-04-25T11:00:39.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.714+0000 7efce0826640 1 -- 172.21.15.69:0/1302246735 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2067405617 0 0) 0x7efcc8003520 con 0x7efcdc06c5c0 2024-04-25T11:00:39.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.714+0000 7efce0826640 1 -- 172.21.15.69:0/1302246735 >> v2:172.21.15.146:3300/0 conn(0x7efcdc0d3000 msgr2=0x7efcdc0cef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:39.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.714+0000 7efce0826640 1 --2- 172.21.15.69:0/1302246735 >> v2:172.21.15.146:3300/0 conn(0x7efcdc0d3000 0x7efcdc0cef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:39.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.714+0000 7efce0826640 1 -- 172.21.15.69:0/1302246735 >> v1:172.21.15.146:6789/0 conn(0x7efcdc15e3b0 legacy=0x7efcdc0d28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:39.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.714+0000 7efce0826640 1 -- 172.21.15.69:0/1302246735 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcdc0b3ed0 con 0x7efcdc06c5c0 2024-04-25T11:00:39.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.714+0000 7efce0826640 1 -- 172.21.15.69:0/1302246735 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7efcd80017f0 con 0x7efcdc06c5c0 2024-04-25T11:00:39.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.714+0000 7efce0826640 1 -- 172.21.15.69:0/1302246735 >> v1:172.21.15.69:6789/0 conn(0x7efcdc06c5c0 legacy=0x7efcdc0cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:39.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.714+0000 7efce0826640 1 --2- 172.21.15.69:0/1302246735 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcc80039b0 0x7efcc8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:39.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.714+0000 7efce0826640 1 --2- 172.21.15.69:0/1302246735 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efcc8004800 0x7efcc8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:39.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.714+0000 7efce0826640 1 --2- 172.21.15.69:0/1302246735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcc80051c0 0x7efcc800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:39.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.714+0000 7efce0826640 1 -- 172.21.15.69:0/1302246735 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efcdc0b3bf0 con 0x7efcc80051c0 2024-04-25T11:00:39.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.714+0000 7efce0826640 1 -- 172.21.15.69:0/1302246735 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7efcc800c160 con 0x7efcc80039b0 2024-04-25T11:00:39.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.714+0000 7efce0826640 1 -- 172.21.15.69:0/1302246735 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7efcc800c360 con 0x7efcc8004800 2024-04-25T11:00:39.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.714+0000 7efce2029640 1 --2- 172.21.15.69:0/1302246735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcc80051c0 0x7efcc800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:39.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.714+0000 7efce1828640 1 --2- 172.21.15.69:0/1302246735 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcc80039b0 0x7efcc8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:39.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.714+0000 7efce1027640 1 --2- 172.21.15.69:0/1302246735 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efcc8004800 0x7efcc8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:39.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.714+0000 7efce2029640 1 -- 172.21.15.69:0/1302246735 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efcc8004800 msgr2=0x7efcc8004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:39.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.714+0000 7efce2029640 1 --2- 172.21.15.69:0/1302246735 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efcc8004800 0x7efcc8004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:39.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.714+0000 7efce2029640 1 -- 172.21.15.69:0/1302246735 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcc80039b0 msgr2=0x7efcc8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:39.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.714+0000 7efce2029640 1 --2- 172.21.15.69:0/1302246735 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcc80039b0 0x7efcc8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:39.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.714+0000 7efce2029640 1 -- 172.21.15.69:0/1302246735 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc8003520 con 0x7efcc80051c0 2024-04-25T11:00:39.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.715+0000 7efce2029640 1 -- 172.21.15.69:0/1302246735 reap_dead start 2024-04-25T11:00:39.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.715+0000 7efce2029640 1 --2- 172.21.15.69:0/1302246735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcc80051c0 0x7efcc800b9c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7efcd4009f90 tx=0x7efcc8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c10f1d528a73b31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:39.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.715+0000 7efce0826640 1 -- 172.21.15.69:0/1302246735 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd4037070 con 0x7efcc80051c0 2024-04-25T11:00:39.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.715+0000 7efce0826640 1 -- 172.21.15.69:0/1302246735 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcd4032620 con 0x7efcc80051c0 2024-04-25T11:00:39.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.715+0000 7efce0826640 1 -- 172.21.15.69:0/1302246735 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd4009c10 con 0x7efcc80051c0 2024-04-25T11:00:39.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.716+0000 7efce3a8c640 1 -- 172.21.15.69:0/1302246735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcc80051c0 msgr2=0x7efcc800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:39.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.716+0000 7efce3a8c640 1 --2- 172.21.15.69:0/1302246735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcc80051c0 0x7efcc800b9c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7efcd4009f90 tx=0x7efcc8005830 comp rx=0 tx=0).stop 2024-04-25T11:00:39.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.716+0000 7efce3a8c640 1 -- 172.21.15.69:0/1302246735 shutdown_connections 2024-04-25T11:00:39.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.716+0000 7efce3a8c640 1 --2- 172.21.15.69:0/1302246735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcc80051c0 0x7efcc800b9c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:39.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.716+0000 7efce3a8c640 1 -- 172.21.15.69:0/1302246735 >> 172.21.15.69:0/1302246735 conn(0x7efcdc09b7d0 msgr2=0x7efcdc09bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:39.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.716+0000 7efce3a8c640 1 -- 172.21.15.69:0/1302246735 shutdown_connections 2024-04-25T11:00:39.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.716+0000 7efce3a8c640 1 -- 172.21.15.69:0/1302246735 wait complete. 2024-04-25T11:00:39.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.718+0000 7efce3a8c640 1 Processor -- start 2024-04-25T11:00:39.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.718+0000 7efce3a8c640 1 -- start start 2024-04-25T11:00:39.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.719+0000 7efce3a8c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efcc80051c0 0x7efcdc14df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:39.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.719+0000 7efce3a8c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcdc14e730 0x7efcdc160440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:39.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.719+0000 7efce3a8c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcdc14ec80 0x7efcdc160980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:39.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.719+0000 7efce3a8c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efcdc0cf290 con 0x7efcdc14ec80 2024-04-25T11:00:39.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.719+0000 7efce3a8c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7efcdc0cf110 con 0x7efcdc14e730 2024-04-25T11:00:39.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.719+0000 7efce3a8c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7efcdc0cf6a0 con 0x7efcc80051c0 2024-04-25T11:00:39.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.719+0000 7efce2029640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcdc14ec80 0x7efcdc160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:39.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.719+0000 7efce2029640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcdc14ec80 0x7efcdc160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47930/0 (socket says 172.21.15.69:47930) 2024-04-25T11:00:39.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.719+0000 7efce2029640 1 -- 172.21.15.69:0/3147022417 learned_addr learned my addr 172.21.15.69:0/3147022417 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:39.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.719+0000 7efce1828640 1 --2- 172.21.15.69:0/3147022417 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efcc80051c0 0x7efcdc14df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:39.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.719+0000 7efce1027640 1 --2- 172.21.15.69:0/3147022417 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcdc14e730 0x7efcdc160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:39.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.719+0000 7efce2029640 1 -- 172.21.15.69:0/3147022417 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efcc80051c0 msgr2=0x7efcdc14df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:39.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.719+0000 7efce2029640 1 --2- 172.21.15.69:0/3147022417 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efcc80051c0 0x7efcdc14df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:39.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.719+0000 7efce2029640 1 -- 172.21.15.69:0/3147022417 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcdc14e730 msgr2=0x7efcdc160440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:39.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.719+0000 7efce2029640 1 --2- 172.21.15.69:0/3147022417 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcdc14e730 0x7efcdc160440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:39.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.720+0000 7efce2029640 1 -- 172.21.15.69:0/3147022417 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcdc0c9420 con 0x7efcdc14ec80 2024-04-25T11:00:39.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.720+0000 7efce1027640 1 --2- 172.21.15.69:0/3147022417 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcdc14e730 0x7efcdc160440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:00:39.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.720+0000 7efce2029640 1 --2- 172.21.15.69:0/3147022417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcdc14ec80 0x7efcdc160980 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7efcd402ae60 tx=0x7efcd4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:39.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.720+0000 7efcd27fc640 1 -- 172.21.15.69:0/3147022417 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd4037070 con 0x7efcdc14ec80 2024-04-25T11:00:39.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.720+0000 7efce3a8c640 1 -- 172.21.15.69:0/3147022417 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcdc1616d0 con 0x7efcdc14ec80 2024-04-25T11:00:39.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.720+0000 7efcd27fc640 1 -- 172.21.15.69:0/3147022417 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcd4044070 con 0x7efcdc14ec80 2024-04-25T11:00:39.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.720+0000 7efcd27fc640 1 -- 172.21.15.69:0/3147022417 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd403e8b0 con 0x7efcdc14ec80 2024-04-25T11:00:39.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.720+0000 7efce3a8c640 1 -- 172.21.15.69:0/3147022417 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcdc161110 con 0x7efcdc14ec80 2024-04-25T11:00:39.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.721+0000 7efce3a8c640 1 -- 172.21.15.69:0/3147022417 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcdc050740 con 0x7efcdc14ec80 2024-04-25T11:00:39.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.721+0000 7efcd27fc640 1 -- 172.21.15.69:0/3147022417 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efcd403ea50 con 0x7efcdc14ec80 2024-04-25T11:00:39.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.722+0000 7efcd27fc640 1 --2- 172.21.15.69:0/3147022417 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7efcc00744b0 0x7efcc00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:39.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.722+0000 7efce1828640 1 --2- 172.21.15.69:0/3147022417 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7efcc00744b0 0x7efcc00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:39.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.722+0000 7efcd27fc640 1 -- 172.21.15.69:0/3147022417 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7efcd40b60d0 con 0x7efcdc14ec80 2024-04-25T11:00:39.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.722+0000 7efce1828640 1 --2- 172.21.15.69:0/3147022417 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7efcc00744b0 0x7efcc00769c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efcd8000ef0 tx=0x7efcd8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:39.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.726+0000 7efcd27fc640 1 -- 172.21.15.69:0/3147022417 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efcdc0c9420 con 0x7efcdc14ec80 2024-04-25T11:00:39.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.899+0000 7efce3a8c640 1 -- 172.21.15.69:0/3147022417 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efcdc0cf480 con 0x7efcdc14ec80 2024-04-25T11:00:39.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.900+0000 7efcd27fc640 1 -- 172.21.15.69:0/3147022417 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v32) ==== 100+0+11 (secure 0 0 0) 0x7efcd4080810 con 0x7efcdc14ec80 2024-04-25T11:00:39.900 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-25T11:00:39.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.901+0000 7efce3a8c640 1 -- 172.21.15.69:0/3147022417 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7efcc00744b0 msgr2=0x7efcc00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:39.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.901+0000 7efce3a8c640 1 --2- 172.21.15.69:0/3147022417 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7efcc00744b0 0x7efcc00769c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efcd8000ef0 tx=0x7efcd8009040 comp rx=0 tx=0).stop 2024-04-25T11:00:39.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.901+0000 7efce3a8c640 1 -- 172.21.15.69:0/3147022417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcdc14ec80 msgr2=0x7efcdc160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:39.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.901+0000 7efce3a8c640 1 --2- 172.21.15.69:0/3147022417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcdc14ec80 0x7efcdc160980 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7efcd402ae60 tx=0x7efcd4009750 comp rx=0 tx=0).stop 2024-04-25T11:00:39.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.901+0000 7efce3a8c640 1 -- 172.21.15.69:0/3147022417 shutdown_connections 2024-04-25T11:00:39.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.901+0000 7efce3a8c640 1 --2- 172.21.15.69:0/3147022417 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7efcc00744b0 0x7efcc00769c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:39.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.901+0000 7efce3a8c640 1 --2- 172.21.15.69:0/3147022417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcdc14ec80 0x7efcdc160980 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:39.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.901+0000 7efce3a8c640 1 --2- 172.21.15.69:0/3147022417 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcdc14e730 0x7efcdc160440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:39.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.901+0000 7efce3a8c640 1 --2- 172.21.15.69:0/3147022417 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efcc80051c0 0x7efcdc14df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:39.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.901+0000 7efce3a8c640 1 -- 172.21.15.69:0/3147022417 >> 172.21.15.69:0/3147022417 conn(0x7efcdc09b7d0 msgr2=0x7efcdc158500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:39.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.902+0000 7efce3a8c640 1 -- 172.21.15.69:0/3147022417 shutdown_connections 2024-04-25T11:00:39.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:39.902+0000 7efce3a8c640 1 -- 172.21.15.69:0/3147022417 wait complete. 2024-04-25T11:00:39.913 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:00:40.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.020+0000 7f5ab899d640 1 Processor -- start 2024-04-25T11:00:40.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.020+0000 7f5ab899d640 1 -- start start 2024-04-25T11:00:40.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.021+0000 7f5ab899d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5ab40b1ba0 0x7f5ab4160200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:40.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.021+0000 7f5ab899d640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ab40c8f70 con 0x7f5ab40d1f80 2024-04-25T11:00:40.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.021+0000 7f5ab899d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ab40c9250 con 0x7f5ab4071780 2024-04-25T11:00:40.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.021+0000 7f5ab899d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5ab40c9530 con 0x7f5ab40b1ba0 2024-04-25T11:00:40.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.021+0000 7f5ab2ffd640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f5ab40d1f80 0x7f5ab40c7640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:39480/0 (socket says 172.21.15.69:39480) 2024-04-25T11:00:40.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.021+0000 7f5ab2ffd640 1 -- 172.21.15.69:0/1617262645 learned_addr learned my addr 172.21.15.69:0/1617262645 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:40.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.021+0000 7f5ab37fe640 1 --2- 172.21.15.69:0/1617262645 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5ab40b1ba0 0x7f5ab4160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:40.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.022+0000 7f5ab27fc640 1 -- 172.21.15.69:0/1617262645 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3213116559 0 0) 0x7f5ab40c8f70 con 0x7f5ab40d1f80 2024-04-25T11:00:40.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.022+0000 7f5ab27fc640 1 -- 172.21.15.69:0/1617262645 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a9c003520 con 0x7f5ab40d1f80 2024-04-25T11:00:40.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.022+0000 7f5ab27fc640 1 -- 172.21.15.69:0/1617262645 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 951249352 0 0) 0x7f5ab40c9250 con 0x7f5ab4071780 2024-04-25T11:00:40.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.022+0000 7f5ab37fe640 1 -- 172.21.15.69:0/1617262645 >> v1:172.21.15.146:6789/0 conn(0x7f5ab4071780 legacy=0x7f5ab4071b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:40.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.022+0000 7f5ab37fe640 1 -- 172.21.15.69:0/1617262645 >> v1:172.21.15.69:6789/0 conn(0x7f5ab40d1f80 legacy=0x7f5ab40c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:40.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.022+0000 7f5ab37fe640 1 -- 172.21.15.69:0/1617262645 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab40d24f0 con 0x7f5ab40b1ba0 2024-04-25T11:00:40.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.023+0000 7f5ab37fe640 1 --2- 172.21.15.69:0/1617262645 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5ab40b1ba0 0x7f5ab4160200 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5aa40097d0 tx=0x7f5aa402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49b89c0d473914f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:40.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.023+0000 7f5ab27fc640 1 -- 172.21.15.69:0/1617262645 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4037070 con 0x7f5ab40b1ba0 2024-04-25T11:00:40.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.023+0000 7f5ab27fc640 1 -- 172.21.15.69:0/1617262645 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aa402ae50 con 0x7f5ab40b1ba0 2024-04-25T11:00:40.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.023+0000 7f5ab27fc640 1 -- 172.21.15.69:0/1617262645 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4032640 con 0x7f5ab40b1ba0 2024-04-25T11:00:40.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.023+0000 7f5ab899d640 1 -- 172.21.15.69:0/1617262645 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5ab40b1ba0 msgr2=0x7f5ab4160200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:40.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.023+0000 7f5ab899d640 1 --2- 172.21.15.69:0/1617262645 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5ab40b1ba0 0x7f5ab4160200 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5aa40097d0 tx=0x7f5aa402a100 comp rx=0 tx=0).stop 2024-04-25T11:00:40.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.023+0000 7f5ab899d640 1 -- 172.21.15.69:0/1617262645 shutdown_connections 2024-04-25T11:00:40.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.023+0000 7f5ab899d640 1 --2- 172.21.15.69:0/1617262645 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5ab40b1ba0 0x7f5ab4160200 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:40.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.023+0000 7f5ab899d640 1 -- 172.21.15.69:0/1617262645 >> 172.21.15.69:0/1617262645 conn(0x7f5ab40cf240 msgr2=0x7f5ab40cf650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:40.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.023+0000 7f5ab899d640 1 -- 172.21.15.69:0/1617262645 shutdown_connections 2024-04-25T11:00:40.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.023+0000 7f5ab899d640 1 -- 172.21.15.69:0/1617262645 wait complete. 2024-04-25T11:00:40.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.025+0000 7f5ab899d640 1 Processor -- start 2024-04-25T11:00:40.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.025+0000 7f5ab899d640 1 -- start start 2024-04-25T11:00:40.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.027+0000 7f5ab899d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5ab4071780 0x7f5ab40b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:40.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.027+0000 7f5ab899d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5ab40b1ba0 0x7f5ab40b5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:40.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.027+0000 7f5ab899d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ab40d1f80 0x7f5ab40cba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:40.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.027+0000 7f5ab899d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab40d0c90 con 0x7f5ab40d1f80 2024-04-25T11:00:40.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.027+0000 7f5ab899d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab40d0b10 con 0x7f5ab4071780 2024-04-25T11:00:40.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.027+0000 7f5ab899d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5ab40d0e10 con 0x7f5ab40b1ba0 2024-04-25T11:00:40.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.027+0000 7f5ab3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ab40d1f80 0x7f5ab40cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:40.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.027+0000 7f5ab3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ab40d1f80 0x7f5ab40cba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47944/0 (socket says 172.21.15.69:47944) 2024-04-25T11:00:40.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.027+0000 7f5ab3fff640 1 -- 172.21.15.69:0/2943646321 learned_addr learned my addr 172.21.15.69:0/2943646321 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:40.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.027+0000 7f5ab37fe640 1 --2- 172.21.15.69:0/2943646321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5ab4071780 0x7f5ab40b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:40.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.027+0000 7f5ab2ffd640 1 --2- 172.21.15.69:0/2943646321 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5ab40b1ba0 0x7f5ab40b5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:40.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.027+0000 7f5ab3fff640 1 -- 172.21.15.69:0/2943646321 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5ab40b1ba0 msgr2=0x7f5ab40b5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:40.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.027+0000 7f5ab3fff640 1 --2- 172.21.15.69:0/2943646321 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5ab40b1ba0 0x7f5ab40b5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:40.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.027+0000 7f5ab3fff640 1 -- 172.21.15.69:0/2943646321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5ab4071780 msgr2=0x7f5ab40b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:40.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.027+0000 7f5ab3fff640 1 --2- 172.21.15.69:0/2943646321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5ab4071780 0x7f5ab40b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:40.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.027+0000 7f5ab3fff640 1 -- 172.21.15.69:0/2943646321 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab40b6410 con 0x7f5ab40d1f80 2024-04-25T11:00:40.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.027+0000 7f5ab3fff640 1 --2- 172.21.15.69:0/2943646321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ab40d1f80 0x7f5ab40cba40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00c780 tx=0x7f5aac00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:40.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.027+0000 7f5a9bfff640 1 -- 172.21.15.69:0/2943646321 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac003b70 con 0x7f5ab40d1f80 2024-04-25T11:00:40.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.027+0000 7f5a9bfff640 1 -- 172.21.15.69:0/2943646321 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aac003d10 con 0x7f5ab40d1f80 2024-04-25T11:00:40.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.027+0000 7f5ab899d640 1 -- 172.21.15.69:0/2943646321 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab40b6920 con 0x7f5ab40d1f80 2024-04-25T11:00:40.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.027+0000 7f5a9bfff640 1 -- 172.21.15.69:0/2943646321 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac005270 con 0x7f5ab40d1f80 2024-04-25T11:00:40.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.028+0000 7f5ab899d640 1 -- 172.21.15.69:0/2943646321 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab40b6ee0 con 0x7f5ab40d1f80 2024-04-25T11:00:40.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.029+0000 7f5ab899d640 1 -- 172.21.15.69:0/2943646321 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ab40b6670 con 0x7f5ab40d1f80 2024-04-25T11:00:40.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.029+0000 7f5a9bfff640 1 -- 172.21.15.69:0/2943646321 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5aac005410 con 0x7f5ab40d1f80 2024-04-25T11:00:40.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.029+0000 7f5a9bfff640 1 --2- 172.21.15.69:0/2943646321 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5aa00744b0 0x7f5aa00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:40.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.030+0000 7f5a9bfff640 1 -- 172.21.15.69:0/2943646321 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7f5aac094fa0 con 0x7f5ab40d1f80 2024-04-25T11:00:40.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.030+0000 7f5ab37fe640 1 --2- 172.21.15.69:0/2943646321 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5aa00744b0 0x7f5aa00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:40.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.030+0000 7f5ab37fe640 1 --2- 172.21.15.69:0/2943646321 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5aa00744b0 0x7f5aa00769c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4034000 tx=0x7f5aa402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:40.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.034+0000 7f5a9bfff640 1 -- 172.21.15.69:0/2943646321 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5aac05f110 con 0x7f5ab40d1f80 2024-04-25T11:00:40.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.245+0000 7f5ab899d640 1 -- 172.21.15.69:0/2943646321 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ab40d1880 con 0x7f5ab40d1f80 2024-04-25T11:00:40.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.246+0000 7f5a9bfff640 1 -- 172.21.15.69:0/2943646321 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+5180 (secure 0 0 0) 0x7f5aac063410 con 0x7f5ab40d1f80 2024-04-25T11:00:40.247 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:00:40.247 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2223802951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":2223802951},{"type":"v1","addr":"172.21.15.146:6835","nonce":2223802951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4867,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/8243043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":8243043},{"type":"v1","addr":"172.21.15.146:6834","nonce":8243043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5250,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1639802532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1639802532},{"type":"v1","addr":"172.21.15.69:6837","nonce":1639802532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:00:06.608766+0000","modified":"2024-04-25T11:00:34.079108+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5031":{"gid":5031,"name":"c","rank":0,"incarnation":17,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6836/3766456547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3766456547},{"type":"v1","addr":"172.21.15.69:6836","nonce":3766456547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5031,"qdb_cluster":[5031]},"id":2}]} 2024-04-25T11:00:40.247 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 21 2024-04-25T11:00:40.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.248+0000 7f5ab899d640 1 -- 172.21.15.69:0/2943646321 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5aa00744b0 msgr2=0x7f5aa00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:40.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.248+0000 7f5ab899d640 1 --2- 172.21.15.69:0/2943646321 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5aa00744b0 0x7f5aa00769c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4034000 tx=0x7f5aa402e000 comp rx=0 tx=0).stop 2024-04-25T11:00:40.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.248+0000 7f5ab899d640 1 -- 172.21.15.69:0/2943646321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ab40d1f80 msgr2=0x7f5ab40cba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:40.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.248+0000 7f5ab899d640 1 --2- 172.21.15.69:0/2943646321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ab40d1f80 0x7f5ab40cba40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00c780 tx=0x7f5aac00cb80 comp rx=0 tx=0).stop 2024-04-25T11:00:40.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.248+0000 7f5ab899d640 1 -- 172.21.15.69:0/2943646321 shutdown_connections 2024-04-25T11:00:40.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.248+0000 7f5ab899d640 1 --2- 172.21.15.69:0/2943646321 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5aa00744b0 0x7f5aa00769c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:40.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.248+0000 7f5ab899d640 1 --2- 172.21.15.69:0/2943646321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ab40d1f80 0x7f5ab40cba40 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:40.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.248+0000 7f5ab899d640 1 --2- 172.21.15.69:0/2943646321 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5ab40b1ba0 0x7f5ab40b5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:40.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.248+0000 7f5ab899d640 1 --2- 172.21.15.69:0/2943646321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5ab4071780 0x7f5ab40b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:40.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.248+0000 7f5ab899d640 1 -- 172.21.15.69:0/2943646321 >> 172.21.15.69:0/2943646321 conn(0x7f5ab40cf240 msgr2=0x7f5ab40b37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:40.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.248+0000 7f5ab899d640 1 -- 172.21.15.69:0/2943646321 shutdown_connections 2024-04-25T11:00:40.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.248+0000 7f5ab899d640 1 -- 172.21.15.69:0/2943646321 wait complete. 2024-04-25T11:00:40.260 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:00:40.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.367+0000 7f053c453640 1 Processor -- start 2024-04-25T11:00:40.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.367+0000 7f053c453640 1 -- start start 2024-04-25T11:00:40.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.368+0000 7f053c453640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0534160510 0x7f053406f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:40.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.368+0000 7f053c453640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f05341645e0 0x7f053406d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:40.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.368+0000 7f053c453640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05340cce10 con 0x7f053406f910 2024-04-25T11:00:40.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.368+0000 7f053c453640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f05340cd0b0 con 0x7f05341645e0 2024-04-25T11:00:40.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.368+0000 7f053c453640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f0534164aa0 con 0x7f0534160510 2024-04-25T11:00:40.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.368+0000 7f053a9f0640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f05341645e0 0x7f053406d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:40.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.368+0000 7f05399ee640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f053406f910 0x7f0534163ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:39494/0 (socket says 172.21.15.69:39494) 2024-04-25T11:00:40.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.368+0000 7f053a9f0640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f05341645e0 0x7f053406d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47954/0 (socket says 172.21.15.69:47954) 2024-04-25T11:00:40.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.368+0000 7f05399ee640 1 -- 172.21.15.69:0/3534637909 learned_addr learned my addr 172.21.15.69:0/3534637909 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:40.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.369+0000 7f053a1ef640 1 --2- 172.21.15.69:0/3534637909 >> v2:172.21.15.146:3300/0 conn(0x7f0534160510 0x7f053406f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:40.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.369+0000 7f053a9f0640 1 -- 172.21.15.69:0/3534637909 >> v2:172.21.15.146:3300/0 conn(0x7f0534160510 msgr2=0x7f053406f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:40.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.369+0000 7f053a9f0640 1 --2- 172.21.15.69:0/3534637909 >> v2:172.21.15.146:3300/0 conn(0x7f0534160510 0x7f053406f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:40.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.369+0000 7f053a9f0640 1 -- 172.21.15.69:0/3534637909 >> v1:172.21.15.69:6789/0 conn(0x7f053406f910 legacy=0x7f0534163ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:40.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.369+0000 7f053a9f0640 1 -- 172.21.15.69:0/3534637909 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0534164d80 con 0x7f05341645e0 2024-04-25T11:00:40.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.369+0000 7f05391ed640 1 -- 172.21.15.69:0/3534637909 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1615289667 0 0) 0x7f05340cce10 con 0x7f053406f910 2024-04-25T11:00:40.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.369+0000 7f053a9f0640 1 --2- 172.21.15.69:0/3534637909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05341645e0 0x7f053406d550 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0534160920 tx=0x7f052c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60138c9a469b1d4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:40.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.370+0000 7f05391ed640 1 -- 172.21.15.69:0/3534637909 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c007740 con 0x7f05341645e0 2024-04-25T11:00:40.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.370+0000 7f05391ed640 1 -- 172.21.15.69:0/3534637909 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f052c0078e0 con 0x7f05341645e0 2024-04-25T11:00:40.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.370+0000 7f05391ed640 1 -- 172.21.15.69:0/3534637909 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c007cb0 con 0x7f05341645e0 2024-04-25T11:00:40.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.370+0000 7f053c453640 1 -- 172.21.15.69:0/3534637909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05341645e0 msgr2=0x7f053406d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:40.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.370+0000 7f053c453640 1 --2- 172.21.15.69:0/3534637909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05341645e0 0x7f053406d550 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0534160920 tx=0x7f052c02a680 comp rx=0 tx=0).stop 2024-04-25T11:00:40.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.370+0000 7f053c453640 1 -- 172.21.15.69:0/3534637909 shutdown_connections 2024-04-25T11:00:40.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.370+0000 7f053c453640 1 --2- 172.21.15.69:0/3534637909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05341645e0 0x7f053406d550 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:40.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.370+0000 7f053c453640 1 --2- 172.21.15.69:0/3534637909 >> v2:172.21.15.146:3300/0 conn(0x7f0534160510 0x7f053406f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:40.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.370+0000 7f053c453640 1 -- 172.21.15.69:0/3534637909 >> 172.21.15.69:0/3534637909 conn(0x7f05340ce790 msgr2=0x7f05340ceba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:40.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.370+0000 7f053c453640 1 -- 172.21.15.69:0/3534637909 shutdown_connections 2024-04-25T11:00:40.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.370+0000 7f053c453640 1 -- 172.21.15.69:0/3534637909 wait complete. 2024-04-25T11:00:40.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.372+0000 7f053c453640 1 Processor -- start 2024-04-25T11:00:40.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.372+0000 7f053c453640 1 -- start start 2024-04-25T11:00:40.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.373+0000 7f053c453640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f053406f910 0x7f0534160920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:40.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.373+0000 7f053c453640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0534160510 0x7f053415aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:40.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.373+0000 7f053c453640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f05341632d0 0x7f05340a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:40.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.373+0000 7f053c453640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f05340d1900 con 0x7f0534160510 2024-04-25T11:00:40.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.373+0000 7f053c453640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f05340d1780 con 0x7f053406f910 2024-04-25T11:00:40.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.373+0000 7f053c453640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f05340d1a80 con 0x7f05341632d0 2024-04-25T11:00:40.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.374+0000 7f05399ee640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0534160510 0x7f053415aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:40.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.374+0000 7f05399ee640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0534160510 0x7f053415aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47958/0 (socket says 172.21.15.69:47958) 2024-04-25T11:00:40.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.374+0000 7f05399ee640 1 -- 172.21.15.69:0/1086585037 learned_addr learned my addr 172.21.15.69:0/1086585037 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:40.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.374+0000 7f053a1ef640 1 --2- 172.21.15.69:0/1086585037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f053406f910 0x7f0534160920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:40.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.374+0000 7f053a9f0640 1 --2- 172.21.15.69:0/1086585037 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f05341632d0 0x7f05340a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:40.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.374+0000 7f05399ee640 1 -- 172.21.15.69:0/1086585037 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f05341632d0 msgr2=0x7f05340a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:40.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.374+0000 7f05399ee640 1 --2- 172.21.15.69:0/1086585037 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f05341632d0 0x7f05340a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:40.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.374+0000 7f05399ee640 1 -- 172.21.15.69:0/1086585037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f053406f910 msgr2=0x7f0534160920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:40.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.374+0000 7f05399ee640 1 --2- 172.21.15.69:0/1086585037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f053406f910 0x7f0534160920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:40.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.374+0000 7f05399ee640 1 -- 172.21.15.69:0/1086585037 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0534160e60 con 0x7f0534160510 2024-04-25T11:00:40.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.374+0000 7f053a1ef640 1 --2- 172.21.15.69:0/1086585037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f053406f910 0x7f0534160920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:40.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.374+0000 7f05399ee640 1 --2- 172.21.15.69:0/1086585037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0534160510 0x7f053415aee0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f052400c410 tx=0x7f052400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:40.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.374+0000 7f052affd640 1 -- 172.21.15.69:0/1086585037 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0524017070 con 0x7f0534160510 2024-04-25T11:00:40.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.374+0000 7f052affd640 1 -- 172.21.15.69:0/1086585037 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0524005ce0 con 0x7f0534160510 2024-04-25T11:00:40.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.374+0000 7f053c453640 1 -- 172.21.15.69:0/1086585037 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05340a2760 con 0x7f0534160510 2024-04-25T11:00:40.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.374+0000 7f052affd640 1 -- 172.21.15.69:0/1086585037 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05240072f0 con 0x7f0534160510 2024-04-25T11:00:40.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.374+0000 7f053c453640 1 -- 172.21.15.69:0/1086585037 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05340a2d20 con 0x7f0534160510 2024-04-25T11:00:40.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.375+0000 7f053c453640 1 -- 172.21.15.69:0/1086585037 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0534050740 con 0x7f0534160510 2024-04-25T11:00:40.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.376+0000 7f052affd640 1 -- 172.21.15.69:0/1086585037 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f052401a070 con 0x7f0534160510 2024-04-25T11:00:40.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.376+0000 7f052affd640 1 --2- 172.21.15.69:0/1086585037 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f05180744e0 0x7f05180769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:40.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.376+0000 7f052affd640 1 -- 172.21.15.69:0/1086585037 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7f0524094570 con 0x7f0534160510 2024-04-25T11:00:40.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.376+0000 7f053a1ef640 1 --2- 172.21.15.69:0/1086585037 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f05180744e0 0x7f05180769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:40.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.377+0000 7f053a1ef640 1 --2- 172.21.15.69:0/1086585037 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f05180744e0 0x7f05180769f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0530006fd0 tx=0x7f0530008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:40.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.381+0000 7f052affd640 1 -- 172.21.15.69:0/1086585037 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f052405e650 con 0x7f0534160510 2024-04-25T11:00:40.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.560+0000 7f053c453640 1 -- 172.21.15.69:0/1086585037 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f05340cdc20 con 0x7f0534160510 2024-04-25T11:00:40.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.561+0000 7f052affd640 1 -- 172.21.15.69:0/1086585037 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16798 (secure 0 0 0) 0x7f0524062950 con 0x7f0534160510 2024-04-25T11:00:40.562 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:00:40.562 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":32,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T11:00:30.066894+0000","last_up_change":"2024-04-25T10:59:05.546839+0000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:59:06.422674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:00:05.007593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T11:00:05.558880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6825","nonce":1876101065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6827","nonce":1876101065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6831","nonce":1876101065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6829","nonce":1876101065}]},"public_addr":"172.21.15.69:6825/1876101065","cluster_addr":"172.21.15.69:6827/1876101065","heartbeat_back_addr":"172.21.15.69:6831/1876101065","heartbeat_front_addr":"172.21.15.69:6829/1876101065","state":["exists","up"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6801","nonce":1806438563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6803","nonce":1806438563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6807","nonce":1806438563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6805","nonce":1806438563}]},"public_addr":"172.21.15.69:6801/1806438563","cluster_addr":"172.21.15.69:6803/1806438563","heartbeat_back_addr":"172.21.15.69:6807/1806438563","heartbeat_front_addr":"172.21.15.69:6805/1806438563","state":["exists","up"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6817","nonce":1593617983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6819","nonce":1593617983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6823","nonce":1593617983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6821","nonce":1593617983}]},"public_addr":"172.21.15.69:6817/1593617983","cluster_addr":"172.21.15.69:6819/1593617983","heartbeat_back_addr":"172.21.15.69:6823/1593617983","heartbeat_front_addr":"172.21.15.69:6821/1593617983","state":["exists","up"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6809","nonce":3950661294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6811","nonce":3950661294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6815","nonce":3950661294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6813","nonce":3950661294}]},"public_addr":"172.21.15.69:6809/3950661294","cluster_addr":"172.21.15.69:6811/3950661294","heartbeat_back_addr":"172.21.15.69:6815/3950661294","heartbeat_front_addr":"172.21.15.69:6813/3950661294","state":["exists","up"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6801","nonce":2508929872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6803","nonce":2508929872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6807","nonce":2508929872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6805","nonce":2508929872}]},"public_addr":"172.21.15.146:6801/2508929872","cluster_addr":"172.21.15.146:6803/2508929872","heartbeat_back_addr":"172.21.15.146:6807/2508929872","heartbeat_front_addr":"172.21.15.146:6805/2508929872","state":["exists","up"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6811","nonce":1310911138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6816","nonce":1310911138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6823","nonce":1310911138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6821","nonce":1310911138}]},"public_addr":"172.21.15.146:6811/1310911138","cluster_addr":"172.21.15.146:6816/1310911138","heartbeat_back_addr":"172.21.15.146:6823/1310911138","heartbeat_front_addr":"172.21.15.146:6821/1310911138","state":["exists","up"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6810","nonce":634688021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6813","nonce":634688021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6820","nonce":634688021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6817","nonce":634688021}]},"public_addr":"172.21.15.146:6810/634688021","cluster_addr":"172.21.15.146:6813/634688021","heartbeat_back_addr":"172.21.15.146:6820/634688021","heartbeat_front_addr":"172.21.15.146:6817/634688021","state":["exists","up"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6825","nonce":3814215935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6827","nonce":3814215935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6831","nonce":3814215935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6829","nonce":3814215935}]},"public_addr":"172.21.15.146:6825/3814215935","cluster_addr":"172.21.15.146:6827/3814215935","heartbeat_back_addr":"172.21.15.146:6831/3814215935","heartbeat_front_addr":"172.21.15.146:6829/3814215935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.820200+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.850189+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.764560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.866248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.807897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.869881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.782310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.827131+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6837/3153221129":"2024-04-26T11:00:30.066863+0000","172.21.15.69:6835/3153221129":"2024-04-26T11:00:30.066863+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:00:40.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.563+0000 7f053c453640 1 -- 172.21.15.69:0/1086585037 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f05180744e0 msgr2=0x7f05180769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:40.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.563+0000 7f053c453640 1 --2- 172.21.15.69:0/1086585037 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f05180744e0 0x7f05180769f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0530006fd0 tx=0x7f0530008040 comp rx=0 tx=0).stop 2024-04-25T11:00:40.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.563+0000 7f053c453640 1 -- 172.21.15.69:0/1086585037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0534160510 msgr2=0x7f053415aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:40.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.563+0000 7f053c453640 1 --2- 172.21.15.69:0/1086585037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0534160510 0x7f053415aee0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f052400c410 tx=0x7f052400c8d0 comp rx=0 tx=0).stop 2024-04-25T11:00:40.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.563+0000 7f053c453640 1 -- 172.21.15.69:0/1086585037 shutdown_connections 2024-04-25T11:00:40.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.563+0000 7f053c453640 1 --2- 172.21.15.69:0/1086585037 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f05180744e0 0x7f05180769f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:40.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.563+0000 7f053c453640 1 --2- 172.21.15.69:0/1086585037 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f05341632d0 0x7f05340a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:40.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.563+0000 7f053c453640 1 --2- 172.21.15.69:0/1086585037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0534160510 0x7f053415aee0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:40.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.563+0000 7f053c453640 1 --2- 172.21.15.69:0/1086585037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f053406f910 0x7f0534160920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:40.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.564+0000 7f053c453640 1 -- 172.21.15.69:0/1086585037 >> 172.21.15.69:0/1086585037 conn(0x7f05340ce790 msgr2=0x7f0534073510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:40.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.564+0000 7f053c453640 1 -- 172.21.15.69:0/1086585037 shutdown_connections 2024-04-25T11:00:40.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.564+0000 7f053c453640 1 -- 172.21.15.69:0/1086585037 wait complete. 2024-04-25T11:00:40.575 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T11:00:40.576 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:00:40.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.682+0000 7f79702d1640 1 Processor -- start 2024-04-25T11:00:40.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.682+0000 7f79702d1640 1 -- start start 2024-04-25T11:00:40.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.683+0000 7f79702d1640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f796815f420 0x7f796815e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:40.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.683+0000 7f79702d1640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79680b3840 0x7f796806c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:40.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.683+0000 7f79702d1640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79680d2220 con 0x7f79680b2d20 2024-04-25T11:00:40.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.683+0000 7f79702d1640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f79680d2df0 con 0x7f796815f420 2024-04-25T11:00:40.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.683+0000 7f79702d1640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f79680d24c0 con 0x7f79680b3840 2024-04-25T11:00:40.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.683+0000 7f796e06d640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f796815f420 0x7f796815e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:40.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.683+0000 7f796e06d640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f796815f420 0x7f796815e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47968/0 (socket says 172.21.15.69:47968) 2024-04-25T11:00:40.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.684+0000 7f796e06d640 1 -- 172.21.15.69:0/3433467851 learned_addr learned my addr 172.21.15.69:0/3433467851 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:40.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.684+0000 7f796e86e640 1 --2- 172.21.15.69:0/3433467851 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79680b3840 0x7f796806c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:40.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.684+0000 7f796e06d640 1 -- 172.21.15.69:0/3433467851 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79680b3840 msgr2=0x7f796806c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:40.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.684+0000 7f796e06d640 1 --2- 172.21.15.69:0/3433467851 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79680b3840 0x7f796806c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:40.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.684+0000 7f796e06d640 1 -- 172.21.15.69:0/3433467851 >> v1:172.21.15.146:6789/0 conn(0x7f79680b2d20 legacy=0x7f79680b3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:40.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.684+0000 7f796e06d640 1 -- 172.21.15.69:0/3433467851 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79680d27a0 con 0x7f796815f420 2024-04-25T11:00:40.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.685+0000 7f796e06d640 1 --2- 172.21.15.69:0/3433467851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f796815f420 0x7f796815e800 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7964009870 tx=0x7f796402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d62a2e8ffe0891cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:40.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.685+0000 7f796d06b640 1 -- 172.21.15.69:0/3433467851 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7964037070 con 0x7f796815f420 2024-04-25T11:00:40.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.685+0000 7f796d06b640 1 -- 172.21.15.69:0/3433467851 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7964032440 con 0x7f796815f420 2024-04-25T11:00:40.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.685+0000 7f796d06b640 1 -- 172.21.15.69:0/3433467851 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7964032810 con 0x7f796815f420 2024-04-25T11:00:40.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.685+0000 7f79702d1640 1 -- 172.21.15.69:0/3433467851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f796815f420 msgr2=0x7f796815e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:40.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.685+0000 7f79702d1640 1 --2- 172.21.15.69:0/3433467851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f796815f420 0x7f796815e800 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7964009870 tx=0x7f796402a1c0 comp rx=0 tx=0).stop 2024-04-25T11:00:40.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.685+0000 7f79702d1640 1 -- 172.21.15.69:0/3433467851 shutdown_connections 2024-04-25T11:00:40.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.685+0000 7f79702d1640 1 --2- 172.21.15.69:0/3433467851 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79680b3840 0x7f796806c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:40.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.685+0000 7f79702d1640 1 --2- 172.21.15.69:0/3433467851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f796815f420 0x7f796815e800 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:40.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.685+0000 7f79702d1640 1 -- 172.21.15.69:0/3433467851 >> 172.21.15.69:0/3433467851 conn(0x7f796809b7d0 msgr2=0x7f79680b7040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:40.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.685+0000 7f79702d1640 1 -- 172.21.15.69:0/3433467851 shutdown_connections 2024-04-25T11:00:40.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.685+0000 7f79702d1640 1 -- 172.21.15.69:0/3433467851 wait complete. 2024-04-25T11:00:40.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.687+0000 7f79702d1640 1 Processor -- start 2024-04-25T11:00:40.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.687+0000 7f79702d1640 1 -- start start 2024-04-25T11:00:40.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.688+0000 7f79702d1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79680b2d20 0x7f7968156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:40.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.688+0000 7f79702d1640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79680b3840 0x7f796815c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:40.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.688+0000 7f79702d1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f796815f420 0x7f79680aeff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:40.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.688+0000 7f79702d1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f79680b78d0 con 0x7f796815f420 2024-04-25T11:00:40.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.688+0000 7f79702d1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f79680b7750 con 0x7f79680b2d20 2024-04-25T11:00:40.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.688+0000 7f79702d1640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f796806cb00 con 0x7f79680b3840 2024-04-25T11:00:40.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.689+0000 7f796e86e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f796815f420 0x7f79680aeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:40.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.689+0000 7f796e86e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f796815f420 0x7f79680aeff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47984/0 (socket says 172.21.15.69:47984) 2024-04-25T11:00:40.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.689+0000 7f796e86e640 1 -- 172.21.15.69:0/2721960020 learned_addr learned my addr 172.21.15.69:0/2721960020 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:40.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.689+0000 7f796e06d640 1 --2- 172.21.15.69:0/2721960020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79680b2d20 0x7f7968156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:40.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.689+0000 7f796d86c640 1 --2- 172.21.15.69:0/2721960020 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79680b3840 0x7f796815c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:40.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.689+0000 7f796e86e640 1 -- 172.21.15.69:0/2721960020 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79680b3840 msgr2=0x7f796815c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:40.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.689+0000 7f796e86e640 1 --2- 172.21.15.69:0/2721960020 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79680b3840 0x7f796815c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:40.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.689+0000 7f796e86e640 1 -- 172.21.15.69:0/2721960020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79680b2d20 msgr2=0x7f7968156b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:40.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.689+0000 7f796e86e640 1 --2- 172.21.15.69:0/2721960020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79680b2d20 0x7f7968156b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:40.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.689+0000 7f796e86e640 1 -- 172.21.15.69:0/2721960020 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79680ce440 con 0x7f796815f420 2024-04-25T11:00:40.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.689+0000 7f796e86e640 1 --2- 172.21.15.69:0/2721960020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f796815f420 0x7f79680aeff0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7960007d40 tx=0x7f796000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:40.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.689+0000 7f795effd640 1 -- 172.21.15.69:0/2721960020 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7960014420 con 0x7f796815f420 2024-04-25T11:00:40.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.689+0000 7f795effd640 1 -- 172.21.15.69:0/2721960020 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7960019070 con 0x7f796815f420 2024-04-25T11:00:40.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.689+0000 7f79702d1640 1 -- 172.21.15.69:0/2721960020 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79680ce720 con 0x7f796815f420 2024-04-25T11:00:40.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.689+0000 7f795effd640 1 -- 172.21.15.69:0/2721960020 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7960014820 con 0x7f796815f420 2024-04-25T11:00:40.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.689+0000 7f79702d1640 1 -- 172.21.15.69:0/2721960020 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79680cece0 con 0x7f796815f420 2024-04-25T11:00:40.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.690+0000 7f79702d1640 1 -- 172.21.15.69:0/2721960020 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7968050740 con 0x7f796815f420 2024-04-25T11:00:40.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.691+0000 7f795effd640 1 -- 172.21.15.69:0/2721960020 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f79600149c0 con 0x7f796815f420 2024-04-25T11:00:40.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.691+0000 7f795effd640 1 --2- 172.21.15.69:0/2721960020 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f794c0744b0 0x7f794c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:40.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.691+0000 7f796e06d640 1 --2- 172.21.15.69:0/2721960020 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f794c0744b0 0x7f794c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:40.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.692+0000 7f795effd640 1 -- 172.21.15.69:0/2721960020 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7f7960095910 con 0x7f796815f420 2024-04-25T11:00:40.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.692+0000 7f796e06d640 1 --2- 172.21.15.69:0/2721960020 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f794c0744b0 0x7f794c0769c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7964034000 tx=0x7f796402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:40.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.696+0000 7f795effd640 1 -- 172.21.15.69:0/2721960020 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f796005f9f0 con 0x7f796815f420 2024-04-25T11:00:40.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.905+0000 7f79702d1640 1 -- 172.21.15.69:0/2721960020 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f796815cad0 con 0x7f796815f420 2024-04-25T11:00:40.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.905+0000 7f795effd640 1 -- 172.21.15.69:0/2721960020 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+5180 (secure 0 0 0) 0x7f7960063cf0 con 0x7f796815f420 2024-04-25T11:00:40.906 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:00:40.906 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2223802951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":2223802951},{"type":"v1","addr":"172.21.15.146:6835","nonce":2223802951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4867,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/8243043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":8243043},{"type":"v1","addr":"172.21.15.146:6834","nonce":8243043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5250,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1639802532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1639802532},{"type":"v1","addr":"172.21.15.69:6837","nonce":1639802532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:00:06.608766+0000","modified":"2024-04-25T11:00:34.079108+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5031":{"gid":5031,"name":"c","rank":0,"incarnation":17,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6836/3766456547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3766456547},{"type":"v1","addr":"172.21.15.69:6836","nonce":3766456547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5031,"qdb_cluster":[5031]},"id":2}]} 2024-04-25T11:00:40.907 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 21 2024-04-25T11:00:40.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.907+0000 7f79702d1640 1 -- 172.21.15.69:0/2721960020 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f794c0744b0 msgr2=0x7f794c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:40.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.907+0000 7f79702d1640 1 --2- 172.21.15.69:0/2721960020 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f794c0744b0 0x7f794c0769c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7964034000 tx=0x7f796402e000 comp rx=0 tx=0).stop 2024-04-25T11:00:40.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.907+0000 7f79702d1640 1 -- 172.21.15.69:0/2721960020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f796815f420 msgr2=0x7f79680aeff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:40.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.907+0000 7f79702d1640 1 --2- 172.21.15.69:0/2721960020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f796815f420 0x7f79680aeff0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7960007d40 tx=0x7f796000e4b0 comp rx=0 tx=0).stop 2024-04-25T11:00:40.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.907+0000 7f79702d1640 1 -- 172.21.15.69:0/2721960020 shutdown_connections 2024-04-25T11:00:40.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.907+0000 7f79702d1640 1 --2- 172.21.15.69:0/2721960020 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f794c0744b0 0x7f794c0769c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:40.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.907+0000 7f79702d1640 1 --2- 172.21.15.69:0/2721960020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f796815f420 0x7f79680aeff0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:40.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.907+0000 7f79702d1640 1 --2- 172.21.15.69:0/2721960020 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79680b3840 0x7f796815c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:40.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.907+0000 7f79702d1640 1 --2- 172.21.15.69:0/2721960020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79680b2d20 0x7f7968156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:40.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.907+0000 7f79702d1640 1 -- 172.21.15.69:0/2721960020 >> 172.21.15.69:0/2721960020 conn(0x7f796809b7d0 msgr2=0x7f79680cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:40.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.907+0000 7f79702d1640 1 -- 172.21.15.69:0/2721960020 shutdown_connections 2024-04-25T11:00:40.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:40.907+0000 7f79702d1640 1 -- 172.21.15.69:0/2721960020 wait complete. 2024-04-25T11:00:40.919 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:00:41.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.025+0000 7efedf706640 1 Processor -- start 2024-04-25T11:00:41.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.025+0000 7efedf706640 1 -- start start 2024-04-25T11:00:41.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.027+0000 7efedf706640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7efed80cfd30 0x7efed80b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:41.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.027+0000 7efedf706640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7efed80d0140 0x7efed80b7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:41.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.027+0000 7efedf706640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efed80cf210 con 0x7efed80d29c0 2024-04-25T11:00:41.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.027+0000 7efedf706640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7efed80b1bc0 con 0x7efed80d0140 2024-04-25T11:00:41.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.027+0000 7efedf706640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7efed80b3c40 con 0x7efed80cfd30 2024-04-25T11:00:41.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.027+0000 7efedcca1640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7efed80d0140 0x7efed80b7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:41.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.027+0000 7efedcca1640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7efed80d0140 0x7efed80b7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47994/0 (socket says 172.21.15.69:47994) 2024-04-25T11:00:41.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.027+0000 7efedcca1640 1 -- 172.21.15.69:0/4114520837 learned_addr learned my addr 172.21.15.69:0/4114520837 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:41.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.027+0000 7efedd4a2640 1 --2- 172.21.15.69:0/4114520837 >> v2:172.21.15.146:3300/0 conn(0x7efed80cfd30 0x7efed80b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:41.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.027+0000 7efedcca1640 1 -- 172.21.15.69:0/4114520837 >> v2:172.21.15.146:3300/0 conn(0x7efed80cfd30 msgr2=0x7efed80b77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:41.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.027+0000 7efedcca1640 1 --2- 172.21.15.69:0/4114520837 >> v2:172.21.15.146:3300/0 conn(0x7efed80cfd30 0x7efed80b77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.027+0000 7efedcca1640 1 -- 172.21.15.69:0/4114520837 >> v1:172.21.15.146:6789/0 conn(0x7efed80d29c0 legacy=0x7efed80d2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:41.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.027+0000 7efedcca1640 1 -- 172.21.15.69:0/4114520837 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed80b3ee0 con 0x7efed80d0140 2024-04-25T11:00:41.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.028+0000 7efedcca1640 1 --2- 172.21.15.69:0/4114520837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efed80d0140 0x7efed80b7d20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7efed80b5530 tx=0x7efec802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50af77ae1cf7fba8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:41.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.028+0000 7efecffff640 1 -- 172.21.15.69:0/4114520837 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec8037070 con 0x7efed80d0140 2024-04-25T11:00:41.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.028+0000 7efecffff640 1 -- 172.21.15.69:0/4114520837 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efec8032440 con 0x7efed80d0140 2024-04-25T11:00:41.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.028+0000 7efecffff640 1 -- 172.21.15.69:0/4114520837 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec8032810 con 0x7efed80d0140 2024-04-25T11:00:41.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.029+0000 7efedf706640 1 -- 172.21.15.69:0/4114520837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efed80d0140 msgr2=0x7efed80b7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:41.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.029+0000 7efedf706640 1 --2- 172.21.15.69:0/4114520837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efed80d0140 0x7efed80b7d20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7efed80b5530 tx=0x7efec802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.029+0000 7efedf706640 1 -- 172.21.15.69:0/4114520837 shutdown_connections 2024-04-25T11:00:41.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.029+0000 7efedf706640 1 --2- 172.21.15.69:0/4114520837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efed80d0140 0x7efed80b7d20 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.029+0000 7efedf706640 1 --2- 172.21.15.69:0/4114520837 >> v2:172.21.15.146:3300/0 conn(0x7efed80cfd30 0x7efed80b77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.029+0000 7efedf706640 1 -- 172.21.15.69:0/4114520837 >> 172.21.15.69:0/4114520837 conn(0x7efed809b7d0 msgr2=0x7efed809bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:41.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.029+0000 7efedf706640 1 -- 172.21.15.69:0/4114520837 shutdown_connections 2024-04-25T11:00:41.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.029+0000 7efedf706640 1 -- 172.21.15.69:0/4114520837 wait complete. 2024-04-25T11:00:41.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.031+0000 7efedf706640 1 Processor -- start 2024-04-25T11:00:41.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.031+0000 7efedf706640 1 -- start start 2024-04-25T11:00:41.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.032+0000 7efedf706640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efed80cfd30 0x7efed815b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:41.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.032+0000 7efedf706640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efed80d0140 0x7efed80c6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:41.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.032+0000 7efedf706640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efed80d29c0 0x7efed80c8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:41.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.032+0000 7efedf706640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efed80b20f0 con 0x7efed80d0140 2024-04-25T11:00:41.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.032+0000 7efedf706640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7efed80b1e10 con 0x7efed80d29c0 2024-04-25T11:00:41.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.032+0000 7efedf706640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7efed81608f0 con 0x7efed80cfd30 2024-04-25T11:00:41.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.032+0000 7efedcca1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efed80d0140 0x7efed80c6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:41.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.032+0000 7efedcca1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efed80d0140 0x7efed80c6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47996/0 (socket says 172.21.15.69:47996) 2024-04-25T11:00:41.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.032+0000 7efedcca1640 1 -- 172.21.15.69:0/1352651930 learned_addr learned my addr 172.21.15.69:0/1352651930 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:41.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.032+0000 7efedd4a2640 1 --2- 172.21.15.69:0/1352651930 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efed80cfd30 0x7efed815b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:41.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.032+0000 7efeddca3640 1 --2- 172.21.15.69:0/1352651930 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efed80d29c0 0x7efed80c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:41.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.032+0000 7efedcca1640 1 -- 172.21.15.69:0/1352651930 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efed80cfd30 msgr2=0x7efed815b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:41.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.032+0000 7efedcca1640 1 --2- 172.21.15.69:0/1352651930 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efed80cfd30 0x7efed815b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.032+0000 7efedcca1640 1 -- 172.21.15.69:0/1352651930 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efed80d29c0 msgr2=0x7efed80c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:41.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.032+0000 7efedcca1640 1 --2- 172.21.15.69:0/1352651930 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efed80d29c0 0x7efed80c8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.032+0000 7efedcca1640 1 -- 172.21.15.69:0/1352651930 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed80c7870 con 0x7efed80d0140 2024-04-25T11:00:41.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.032+0000 7efedcca1640 1 --2- 172.21.15.69:0/1352651930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efed80d0140 0x7efed80c6e90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7efec8000c00 tx=0x7efec80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:41.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.033+0000 7efecdffb640 1 -- 172.21.15.69:0/1352651930 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec8041020 con 0x7efed80d0140 2024-04-25T11:00:41.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.033+0000 7efecdffb640 1 -- 172.21.15.69:0/1352651930 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efec80306c0 con 0x7efed80d0140 2024-04-25T11:00:41.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.033+0000 7efedf706640 1 -- 172.21.15.69:0/1352651930 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed80c7b50 con 0x7efed80d0140 2024-04-25T11:00:41.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.033+0000 7efecdffb640 1 -- 172.21.15.69:0/1352651930 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec8037040 con 0x7efed80d0140 2024-04-25T11:00:41.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.033+0000 7efedf706640 1 -- 172.21.15.69:0/1352651930 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efed80c8110 con 0x7efed80d0140 2024-04-25T11:00:41.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.034+0000 7efedf706640 1 -- 172.21.15.69:0/1352651930 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efed8050740 con 0x7efed80d0140 2024-04-25T11:00:41.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.034+0000 7efecdffb640 1 -- 172.21.15.69:0/1352651930 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efec8044020 con 0x7efed80d0140 2024-04-25T11:00:41.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.035+0000 7efecdffb640 1 --2- 172.21.15.69:0/1352651930 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7efebc0744e0 0x7efebc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:41.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.035+0000 7efecdffb640 1 -- 172.21.15.69:0/1352651930 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7efec80b5b10 con 0x7efed80d0140 2024-04-25T11:00:41.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.035+0000 7efedd4a2640 1 --2- 172.21.15.69:0/1352651930 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7efebc0744e0 0x7efebc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:41.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.035+0000 7efedd4a2640 1 --2- 172.21.15.69:0/1352651930 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7efebc0744e0 0x7efebc0769f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efed4006fd0 tx=0x7efed4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:41.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.040+0000 7efecdffb640 1 -- 172.21.15.69:0/1352651930 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efec804b030 con 0x7efed80d0140 2024-04-25T11:00:41.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.253+0000 7efedf706640 1 -- 172.21.15.69:0/1352651930 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efed80ce940 con 0x7efed80d0140 2024-04-25T11:00:41.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.254+0000 7efecdffb640 1 -- 172.21.15.69:0/1352651930 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+5180 (secure 0 0 0) 0x7efec8080240 con 0x7efed80d0140 2024-04-25T11:00:41.255 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:00:41.255 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2223802951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":2223802951},{"type":"v1","addr":"172.21.15.146:6835","nonce":2223802951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4867,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/8243043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":8243043},{"type":"v1","addr":"172.21.15.146:6834","nonce":8243043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5250,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1639802532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1639802532},{"type":"v1","addr":"172.21.15.69:6837","nonce":1639802532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:00:06.608766+0000","modified":"2024-04-25T11:00:34.079108+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5031":{"gid":5031,"name":"c","rank":0,"incarnation":17,"state":"up:active","state_seq":12,"addr":"172.21.15.69:6836/3766456547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3766456547},{"type":"v1","addr":"172.21.15.69:6836","nonce":3766456547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5031,"qdb_cluster":[5031]},"id":2}]} 2024-04-25T11:00:41.255 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 21 2024-04-25T11:00:41.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.256+0000 7efedf706640 1 -- 172.21.15.69:0/1352651930 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7efebc0744e0 msgr2=0x7efebc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:41.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.256+0000 7efedf706640 1 --2- 172.21.15.69:0/1352651930 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7efebc0744e0 0x7efebc0769f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efed4006fd0 tx=0x7efed4008040 comp rx=0 tx=0).stop 2024-04-25T11:00:41.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.256+0000 7efedf706640 1 -- 172.21.15.69:0/1352651930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efed80d0140 msgr2=0x7efed80c6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:41.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.256+0000 7efedf706640 1 --2- 172.21.15.69:0/1352651930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efed80d0140 0x7efed80c6e90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7efec8000c00 tx=0x7efec80046c0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.256+0000 7efedf706640 1 -- 172.21.15.69:0/1352651930 shutdown_connections 2024-04-25T11:00:41.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.256+0000 7efedf706640 1 --2- 172.21.15.69:0/1352651930 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7efebc0744e0 0x7efebc0769f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.256+0000 7efedf706640 1 --2- 172.21.15.69:0/1352651930 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efed80d29c0 0x7efed80c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.256+0000 7efedf706640 1 --2- 172.21.15.69:0/1352651930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efed80d0140 0x7efed80c6e90 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.256+0000 7efedf706640 1 --2- 172.21.15.69:0/1352651930 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efed80cfd30 0x7efed815b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.256+0000 7efedf706640 1 -- 172.21.15.69:0/1352651930 >> 172.21.15.69:0/1352651930 conn(0x7efed809b7d0 msgr2=0x7efed8071000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:41.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.256+0000 7efedf706640 1 -- 172.21.15.69:0/1352651930 shutdown_connections 2024-04-25T11:00:41.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.256+0000 7efedf706640 1 -- 172.21.15.69:0/1352651930 wait complete. 2024-04-25T11:00:41.267 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:00:41.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.374+0000 7f4f42ae7640 1 Processor -- start 2024-04-25T11:00:41.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.374+0000 7f4f42ae7640 1 -- start start 2024-04-25T11:00:41.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.375+0000 7f4f42ae7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f3c0d1620 0x7f4f3c0d1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:41.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.375+0000 7f4f42ae7640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f4f3c163130 0x7f4f3c0b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:41.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.375+0000 7f4f42ae7640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f3c0b0770 con 0x7f4f3c0ce800 2024-04-25T11:00:41.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.375+0000 7f4f42ae7640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f4f3c0b10a0 con 0x7f4f3c163130 2024-04-25T11:00:41.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.375+0000 7f4f42ae7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4f3c0d2100 con 0x7f4f3c0d1620 2024-04-25T11:00:41.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.375+0000 7f4f41084640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f4f3c163130 0x7f4f3c0b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:41.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.375+0000 7f4f41084640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f4f3c163130 0x7f4f3c0b2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48010/0 (socket says 172.21.15.69:48010) 2024-04-25T11:00:41.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.375+0000 7f4f40883640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f4f3c0ce800 0x7f4f3c162a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:39502/0 (socket says 172.21.15.69:39502) 2024-04-25T11:00:41.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.375+0000 7f4f41084640 1 -- 172.21.15.69:0/807870888 learned_addr learned my addr 172.21.15.69:0/807870888 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:41.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.376+0000 7f4f3bfff640 1 --2- 172.21.15.69:0/807870888 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f3c0d1620 0x7f4f3c0d1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:41.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.376+0000 7f4f41084640 1 -- 172.21.15.69:0/807870888 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f3c0d1620 msgr2=0x7f4f3c0d1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:41.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.376+0000 7f4f41084640 1 --2- 172.21.15.69:0/807870888 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f3c0d1620 0x7f4f3c0d1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.376+0000 7f4f41084640 1 -- 172.21.15.69:0/807870888 >> v1:172.21.15.69:6789/0 conn(0x7f4f3c0ce800 legacy=0x7f4f3c162a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:41.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.376+0000 7f4f3b7fe640 1 -- 172.21.15.69:0/807870888 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1362642557 0 0) 0x7f4f3c0b0770 con 0x7f4f3c0ce800 2024-04-25T11:00:41.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.376+0000 7f4f41084640 1 -- 172.21.15.69:0/807870888 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f3c0b0a50 con 0x7f4f3c163130 2024-04-25T11:00:41.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.376+0000 7f4f3bfff640 1 --2- 172.21.15.69:0/807870888 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f3c0d1620 0x7f4f3c0d1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:00:41.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.376+0000 7f4f41084640 1 --2- 172.21.15.69:0/807870888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f3c163130 0x7f4f3c0b2c30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4f34009e50 tx=0x7f4f3402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda0cee15a5ca582 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:41.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.377+0000 7f4f3b7fe640 1 -- 172.21.15.69:0/807870888 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f34036070 con 0x7f4f3c163130 2024-04-25T11:00:41.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.377+0000 7f4f3b7fe640 1 -- 172.21.15.69:0/807870888 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f34031600 con 0x7f4f3c163130 2024-04-25T11:00:41.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.377+0000 7f4f3b7fe640 1 -- 172.21.15.69:0/807870888 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f340319d0 con 0x7f4f3c163130 2024-04-25T11:00:41.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.377+0000 7f4f42ae7640 1 -- 172.21.15.69:0/807870888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f3c163130 msgr2=0x7f4f3c0b2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:41.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.377+0000 7f4f42ae7640 1 --2- 172.21.15.69:0/807870888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f3c163130 0x7f4f3c0b2c30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4f34009e50 tx=0x7f4f3402a5f0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.377+0000 7f4f42ae7640 1 -- 172.21.15.69:0/807870888 shutdown_connections 2024-04-25T11:00:41.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.377+0000 7f4f42ae7640 1 --2- 172.21.15.69:0/807870888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f3c163130 0x7f4f3c0b2c30 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.377+0000 7f4f42ae7640 1 --2- 172.21.15.69:0/807870888 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f3c0d1620 0x7f4f3c0d1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.377+0000 7f4f42ae7640 1 -- 172.21.15.69:0/807870888 >> 172.21.15.69:0/807870888 conn(0x7f4f3c0d3a50 msgr2=0x7f4f3c0d3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:41.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.377+0000 7f4f42ae7640 1 -- 172.21.15.69:0/807870888 shutdown_connections 2024-04-25T11:00:41.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.377+0000 7f4f42ae7640 1 -- 172.21.15.69:0/807870888 wait complete. 2024-04-25T11:00:41.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.379+0000 7f4f42ae7640 1 Processor -- start 2024-04-25T11:00:41.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.379+0000 7f4f42ae7640 1 -- start start 2024-04-25T11:00:41.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.380+0000 7f4f42ae7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f3c0ce800 0x7f4f3c1f51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:41.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.380+0000 7f4f42ae7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f3c0d1620 0x7f4f3c155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:41.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.380+0000 7f4f42ae7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f3c163130 0x7f4f3c158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:41.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.381+0000 7f4f42ae7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c163540 con 0x7f4f3c0d1620 2024-04-25T11:00:41.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.381+0000 7f4f42ae7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c0b51c0 con 0x7f4f3c163130 2024-04-25T11:00:41.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.381+0000 7f4f42ae7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4f3c15f1a0 con 0x7f4f3c0ce800 2024-04-25T11:00:41.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.381+0000 7f4f3bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f3c0d1620 0x7f4f3c155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:41.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.381+0000 7f4f3bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f3c0d1620 0x7f4f3c155f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48018/0 (socket says 172.21.15.69:48018) 2024-04-25T11:00:41.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.381+0000 7f4f3bfff640 1 -- 172.21.15.69:0/2379448304 learned_addr learned my addr 172.21.15.69:0/2379448304 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:41.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.381+0000 7f4f40883640 1 --2- 172.21.15.69:0/2379448304 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f3c0ce800 0x7f4f3c1f51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:41.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.381+0000 7f4f41084640 1 --2- 172.21.15.69:0/2379448304 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f3c163130 0x7f4f3c158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:41.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.381+0000 7f4f3bfff640 1 -- 172.21.15.69:0/2379448304 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f3c0ce800 msgr2=0x7f4f3c1f51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:41.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.381+0000 7f4f3bfff640 1 --2- 172.21.15.69:0/2379448304 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f3c0ce800 0x7f4f3c1f51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.381+0000 7f4f3bfff640 1 -- 172.21.15.69:0/2379448304 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f3c163130 msgr2=0x7f4f3c158fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:41.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.381+0000 7f4f3bfff640 1 --2- 172.21.15.69:0/2379448304 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f3c163130 0x7f4f3c158fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.381+0000 7f4f3bfff640 1 -- 172.21.15.69:0/2379448304 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f3c160640 con 0x7f4f3c0d1620 2024-04-25T11:00:41.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.381+0000 7f4f3bfff640 1 --2- 172.21.15.69:0/2379448304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f3c0d1620 0x7f4f3c155f80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4f2800c7f0 tx=0x7f4f2800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:41.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.381+0000 7f4f397fa640 1 -- 172.21.15.69:0/2379448304 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f28015070 con 0x7f4f3c0d1620 2024-04-25T11:00:41.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.381+0000 7f4f397fa640 1 -- 172.21.15.69:0/2379448304 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f28007820 con 0x7f4f3c0d1620 2024-04-25T11:00:41.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.381+0000 7f4f397fa640 1 -- 172.21.15.69:0/2379448304 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f28007bf0 con 0x7f4f3c0d1620 2024-04-25T11:00:41.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.381+0000 7f4f42ae7640 1 -- 172.21.15.69:0/2379448304 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f3c1f5720 con 0x7f4f3c0d1620 2024-04-25T11:00:41.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.381+0000 7f4f42ae7640 1 -- 172.21.15.69:0/2379448304 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f3c157990 con 0x7f4f3c0d1620 2024-04-25T11:00:41.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.382+0000 7f4f42ae7640 1 -- 172.21.15.69:0/2379448304 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f3c157420 con 0x7f4f3c0d1620 2024-04-25T11:00:41.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.383+0000 7f4f397fa640 1 -- 172.21.15.69:0/2379448304 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4f28018070 con 0x7f4f3c0d1620 2024-04-25T11:00:41.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.383+0000 7f4f397fa640 1 --2- 172.21.15.69:0/2379448304 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4f200744b0 0x7f4f200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:41.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.383+0000 7f4f397fa640 1 -- 172.21.15.69:0/2379448304 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7f4f28095310 con 0x7f4f3c0d1620 2024-04-25T11:00:41.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.383+0000 7f4f40883640 1 --2- 172.21.15.69:0/2379448304 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4f200744b0 0x7f4f200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:41.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.384+0000 7f4f40883640 1 --2- 172.21.15.69:0/2379448304 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4f200744b0 0x7f4f200769c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4f30007fd0 tx=0x7f4f30009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:41.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.388+0000 7f4f397fa640 1 -- 172.21.15.69:0/2379448304 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4f2805fad0 con 0x7f4f3c0d1620 2024-04-25T11:00:41.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.564+0000 7f4f42ae7640 1 -- 172.21.15.69:0/2379448304 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4f3c06d0d0 con 0x7f4f3c0d1620 2024-04-25T11:00:41.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.565+0000 7f4f397fa640 1 -- 172.21.15.69:0/2379448304 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16798 (secure 0 0 0) 0x7f4f3c06d0d0 con 0x7f4f3c0d1620 2024-04-25T11:00:41.566 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:00:41.566 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":32,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T11:00:30.066894+0000","last_up_change":"2024-04-25T10:59:05.546839+0000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:59:06.422674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:00:05.007593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T11:00:05.558880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6825","nonce":1876101065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6827","nonce":1876101065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6831","nonce":1876101065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6829","nonce":1876101065}]},"public_addr":"172.21.15.69:6825/1876101065","cluster_addr":"172.21.15.69:6827/1876101065","heartbeat_back_addr":"172.21.15.69:6831/1876101065","heartbeat_front_addr":"172.21.15.69:6829/1876101065","state":["exists","up"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6801","nonce":1806438563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6803","nonce":1806438563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6807","nonce":1806438563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6805","nonce":1806438563}]},"public_addr":"172.21.15.69:6801/1806438563","cluster_addr":"172.21.15.69:6803/1806438563","heartbeat_back_addr":"172.21.15.69:6807/1806438563","heartbeat_front_addr":"172.21.15.69:6805/1806438563","state":["exists","up"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6817","nonce":1593617983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6819","nonce":1593617983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6823","nonce":1593617983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6821","nonce":1593617983}]},"public_addr":"172.21.15.69:6817/1593617983","cluster_addr":"172.21.15.69:6819/1593617983","heartbeat_back_addr":"172.21.15.69:6823/1593617983","heartbeat_front_addr":"172.21.15.69:6821/1593617983","state":["exists","up"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6809","nonce":3950661294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6811","nonce":3950661294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6815","nonce":3950661294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6813","nonce":3950661294}]},"public_addr":"172.21.15.69:6809/3950661294","cluster_addr":"172.21.15.69:6811/3950661294","heartbeat_back_addr":"172.21.15.69:6815/3950661294","heartbeat_front_addr":"172.21.15.69:6813/3950661294","state":["exists","up"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6801","nonce":2508929872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6803","nonce":2508929872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6807","nonce":2508929872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6805","nonce":2508929872}]},"public_addr":"172.21.15.146:6801/2508929872","cluster_addr":"172.21.15.146:6803/2508929872","heartbeat_back_addr":"172.21.15.146:6807/2508929872","heartbeat_front_addr":"172.21.15.146:6805/2508929872","state":["exists","up"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6811","nonce":1310911138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6816","nonce":1310911138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6823","nonce":1310911138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6821","nonce":1310911138}]},"public_addr":"172.21.15.146:6811/1310911138","cluster_addr":"172.21.15.146:6816/1310911138","heartbeat_back_addr":"172.21.15.146:6823/1310911138","heartbeat_front_addr":"172.21.15.146:6821/1310911138","state":["exists","up"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6810","nonce":634688021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6813","nonce":634688021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6820","nonce":634688021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6817","nonce":634688021}]},"public_addr":"172.21.15.146:6810/634688021","cluster_addr":"172.21.15.146:6813/634688021","heartbeat_back_addr":"172.21.15.146:6820/634688021","heartbeat_front_addr":"172.21.15.146:6817/634688021","state":["exists","up"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6825","nonce":3814215935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6827","nonce":3814215935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6831","nonce":3814215935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6829","nonce":3814215935}]},"public_addr":"172.21.15.146:6825/3814215935","cluster_addr":"172.21.15.146:6827/3814215935","heartbeat_back_addr":"172.21.15.146:6831/3814215935","heartbeat_front_addr":"172.21.15.146:6829/3814215935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.820200+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.850189+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.764560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.866248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.807897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.869881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.782310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.827131+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6837/3153221129":"2024-04-26T11:00:30.066863+0000","172.21.15.69:6835/3153221129":"2024-04-26T11:00:30.066863+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:00:41.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.567+0000 7f4f42ae7640 1 -- 172.21.15.69:0/2379448304 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4f200744b0 msgr2=0x7f4f200769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:41.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.567+0000 7f4f42ae7640 1 --2- 172.21.15.69:0/2379448304 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4f200744b0 0x7f4f200769c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4f30007fd0 tx=0x7f4f30009040 comp rx=0 tx=0).stop 2024-04-25T11:00:41.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.567+0000 7f4f42ae7640 1 -- 172.21.15.69:0/2379448304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f3c0d1620 msgr2=0x7f4f3c155f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:41.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.567+0000 7f4f42ae7640 1 --2- 172.21.15.69:0/2379448304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f3c0d1620 0x7f4f3c155f80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4f2800c7f0 tx=0x7f4f2800ccb0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.567+0000 7f4f42ae7640 1 -- 172.21.15.69:0/2379448304 shutdown_connections 2024-04-25T11:00:41.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.567+0000 7f4f42ae7640 1 --2- 172.21.15.69:0/2379448304 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4f200744b0 0x7f4f200769c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.567+0000 7f4f42ae7640 1 --2- 172.21.15.69:0/2379448304 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f3c163130 0x7f4f3c158fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.567+0000 7f4f42ae7640 1 --2- 172.21.15.69:0/2379448304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f3c0d1620 0x7f4f3c155f80 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.567+0000 7f4f42ae7640 1 --2- 172.21.15.69:0/2379448304 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4f3c0ce800 0x7f4f3c1f51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.567+0000 7f4f42ae7640 1 -- 172.21.15.69:0/2379448304 >> 172.21.15.69:0/2379448304 conn(0x7f4f3c0d3a50 msgr2=0x7f4f3c0d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:41.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.567+0000 7f4f42ae7640 1 -- 172.21.15.69:0/2379448304 shutdown_connections 2024-04-25T11:00:41.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.567+0000 7f4f42ae7640 1 -- 172.21.15.69:0/2379448304 wait complete. 2024-04-25T11:00:41.579 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:00:41.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.687+0000 7f8288f5f640 1 Processor -- start 2024-04-25T11:00:41.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.687+0000 7f8288f5f640 1 -- start start 2024-04-25T11:00:41.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.688+0000 7f8288f5f640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f82840b8ca0 0x7f8284163fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:41.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.688+0000 7f8288f5f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82841644e0 0x7f82841648f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:41.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.688+0000 7f8288f5f640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f82840d1f30 0x7f82840d2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:41.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.688+0000 7f8288f5f640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f82840d35a0 con 0x7f82840b8ca0 2024-04-25T11:00:41.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.688+0000 7f8288f5f640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f828406e8d0 con 0x7f82840d1f30 2024-04-25T11:00:41.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.688+0000 7f8288f5f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f828406ed60 con 0x7f82841644e0 2024-04-25T11:00:41.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.688+0000 7f828259c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f82840b8ca0 0x7f8284163fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:41.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.688+0000 7f828259c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f82840b8ca0 0x7f8284163fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48032/0 (socket says 172.21.15.69:48032) 2024-04-25T11:00:41.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.688+0000 7f828259c640 1 -- 172.21.15.69:0/3331148229 learned_addr learned my addr 172.21.15.69:0/3331148229 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:41.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.688+0000 7f8281d9b640 1 --2- 172.21.15.69:0/3331148229 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82841644e0 0x7f82841648f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:41.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.689+0000 7f8282d9d640 1 --2- 172.21.15.69:0/3331148229 >> v2:172.21.15.146:3300/0 conn(0x7f82840d1f30 0x7f82840d2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:41.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.689+0000 7f828259c640 1 -- 172.21.15.69:0/3331148229 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82841644e0 msgr2=0x7f82841648f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:41.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.689+0000 7f828259c640 1 --2- 172.21.15.69:0/3331148229 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82841644e0 0x7f82841648f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.689+0000 7f828259c640 1 -- 172.21.15.69:0/3331148229 >> v2:172.21.15.146:3300/0 conn(0x7f82840d1f30 msgr2=0x7f82840d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:41.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.689+0000 7f828259c640 1 --2- 172.21.15.69:0/3331148229 >> v2:172.21.15.146:3300/0 conn(0x7f82840d1f30 0x7f82840d2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.689+0000 7f828259c640 1 -- 172.21.15.69:0/3331148229 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82841608b0 con 0x7f82840b8ca0 2024-04-25T11:00:41.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.689+0000 7f8282d9d640 1 --2- 172.21.15.69:0/3331148229 >> v2:172.21.15.146:3300/0 conn(0x7f82840d1f30 0x7f82840d2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:00:41.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.689+0000 7f828259c640 1 --2- 172.21.15.69:0/3331148229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82840b8ca0 0x7f8284163fa0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8284075f80 tx=0x7f827c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f10c477ef8e5db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:41.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.689+0000 7f828159a640 1 -- 172.21.15.69:0/3331148229 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c037070 con 0x7f82840b8ca0 2024-04-25T11:00:41.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.689+0000 7f828159a640 1 -- 172.21.15.69:0/3331148229 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f827c032440 con 0x7f82840b8ca0 2024-04-25T11:00:41.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.689+0000 7f828159a640 1 -- 172.21.15.69:0/3331148229 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c032810 con 0x7f82840b8ca0 2024-04-25T11:00:41.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.690+0000 7f8288f5f640 1 -- 172.21.15.69:0/3331148229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82840b8ca0 msgr2=0x7f8284163fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:41.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.690+0000 7f8288f5f640 1 --2- 172.21.15.69:0/3331148229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82840b8ca0 0x7f8284163fa0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8284075f80 tx=0x7f827c02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.690+0000 7f8288f5f640 1 -- 172.21.15.69:0/3331148229 shutdown_connections 2024-04-25T11:00:41.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.690+0000 7f8288f5f640 1 --2- 172.21.15.69:0/3331148229 >> v2:172.21.15.146:3300/0 conn(0x7f82840d1f30 0x7f82840d2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.690+0000 7f8288f5f640 1 --2- 172.21.15.69:0/3331148229 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82841644e0 0x7f82841648f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.690+0000 7f8288f5f640 1 --2- 172.21.15.69:0/3331148229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82840b8ca0 0x7f8284163fa0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.690+0000 7f8288f5f640 1 -- 172.21.15.69:0/3331148229 >> 172.21.15.69:0/3331148229 conn(0x7f82840d3aa0 msgr2=0x7f82840d3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:41.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.690+0000 7f8288f5f640 1 -- 172.21.15.69:0/3331148229 shutdown_connections 2024-04-25T11:00:41.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.690+0000 7f8288f5f640 1 -- 172.21.15.69:0/3331148229 wait complete. 2024-04-25T11:00:41.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.692+0000 7f8288f5f640 1 Processor -- start 2024-04-25T11:00:41.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.692+0000 7f8288f5f640 1 -- start start 2024-04-25T11:00:41.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.693+0000 7f8288f5f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82840b8ca0 0x7f82840c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:41.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.693+0000 7f8288f5f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82840d1f30 0x7f8284150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:41.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.694+0000 7f8288f5f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82841644e0 0x7f82841eb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:41.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.694+0000 7f8288f5f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f828406e280 con 0x7f82841644e0 2024-04-25T11:00:41.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.694+0000 7f8288f5f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f82840d32c0 con 0x7f82840b8ca0 2024-04-25T11:00:41.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.694+0000 7f8288f5f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f828406e560 con 0x7f82840d1f30 2024-04-25T11:00:41.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.694+0000 7f8282d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82841644e0 0x7f82841eb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:41.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.694+0000 7f8282d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82841644e0 0x7f82841eb290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48044/0 (socket says 172.21.15.69:48044) 2024-04-25T11:00:41.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.694+0000 7f8282d9d640 1 -- 172.21.15.69:0/1783566867 learned_addr learned my addr 172.21.15.69:0/1783566867 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:41.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.694+0000 7f8281d9b640 1 --2- 172.21.15.69:0/1783566867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82840d1f30 0x7f8284150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:41.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.694+0000 7f828259c640 1 --2- 172.21.15.69:0/1783566867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82840b8ca0 0x7f82840c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:41.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.694+0000 7f8282d9d640 1 -- 172.21.15.69:0/1783566867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82840d1f30 msgr2=0x7f8284150f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:41.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.694+0000 7f8282d9d640 1 --2- 172.21.15.69:0/1783566867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82840d1f30 0x7f8284150f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.694+0000 7f8282d9d640 1 -- 172.21.15.69:0/1783566867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82840b8ca0 msgr2=0x7f82840c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:41.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.694+0000 7f8282d9d640 1 --2- 172.21.15.69:0/1783566867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82840b8ca0 0x7f82840c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.694+0000 7f8282d9d640 1 -- 172.21.15.69:0/1783566867 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82840c8720 con 0x7f82841644e0 2024-04-25T11:00:41.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.694+0000 7f8282d9d640 1 --2- 172.21.15.69:0/1783566867 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82841644e0 0x7f82841eb290 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f827400cd10 tx=0x7f827400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:41.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.694+0000 7f827affd640 1 -- 172.21.15.69:0/1783566867 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827400f680 con 0x7f82841644e0 2024-04-25T11:00:41.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.694+0000 7f827affd640 1 -- 172.21.15.69:0/1783566867 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f827400f840 con 0x7f82841644e0 2024-04-25T11:00:41.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.694+0000 7f8288f5f640 1 -- 172.21.15.69:0/1783566867 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82840cac80 con 0x7f82841644e0 2024-04-25T11:00:41.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.694+0000 7f827affd640 1 -- 172.21.15.69:0/1783566867 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827400fc30 con 0x7f82841644e0 2024-04-25T11:00:41.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.694+0000 7f8288f5f640 1 -- 172.21.15.69:0/1783566867 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82840cb240 con 0x7f82841644e0 2024-04-25T11:00:41.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.695+0000 7f8288f5f640 1 -- 172.21.15.69:0/1783566867 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8284050740 con 0x7f82841644e0 2024-04-25T11:00:41.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.696+0000 7f827affd640 1 -- 172.21.15.69:0/1783566867 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f827400e3e0 con 0x7f82841644e0 2024-04-25T11:00:41.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.696+0000 7f827affd640 1 --2- 172.21.15.69:0/1783566867 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f82640744e0 0x7f82640769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:41.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.697+0000 7f827affd640 1 -- 172.21.15.69:0/1783566867 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7f8274095e80 con 0x7f82841644e0 2024-04-25T11:00:41.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.697+0000 7f828259c640 1 --2- 172.21.15.69:0/1783566867 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f82640744e0 0x7f82640769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:41.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.697+0000 7f828259c640 1 --2- 172.21.15.69:0/1783566867 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f82640744e0 0x7f82640769f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f827c034000 tx=0x7f827c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:41.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.701+0000 7f827affd640 1 -- 172.21.15.69:0/1783566867 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8274014030 con 0x7f82841644e0 2024-04-25T11:00:41.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.914+0000 7f8288f5f640 1 -- 172.21.15.69:0/1783566867 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f82840d4700 con 0x7f82841644e0 2024-04-25T11:00:41.915 INFO:tasks.ceph.mon.a.smithi069.stderr:2024-04-25T11:00:41.915+0000 7f5831b98640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:00:41.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.916+0000 7f827affd640 1 -- 172.21.15.69:0/1783566867 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v22) ==== 161+0+0 (secure 0 0 0) 0x7f827405fee0 con 0x7f82841644e0 2024-04-25T11:00:41.917 INFO:teuthology.orchestra.run.smithi069.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:00:41.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.918+0000 7f8288f5f640 1 -- 172.21.15.69:0/1783566867 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f82640744e0 msgr2=0x7f82640769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:41.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.918+0000 7f8288f5f640 1 --2- 172.21.15.69:0/1783566867 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f82640744e0 0x7f82640769f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f827c034000 tx=0x7f827c02e000 comp rx=0 tx=0).stop 2024-04-25T11:00:41.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.918+0000 7f8288f5f640 1 -- 172.21.15.69:0/1783566867 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82841644e0 msgr2=0x7f82841eb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:41.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.918+0000 7f8288f5f640 1 --2- 172.21.15.69:0/1783566867 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82841644e0 0x7f82841eb290 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f827400cd10 tx=0x7f827400a650 comp rx=0 tx=0).stop 2024-04-25T11:00:41.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.918+0000 7f8288f5f640 1 -- 172.21.15.69:0/1783566867 shutdown_connections 2024-04-25T11:00:41.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.918+0000 7f8288f5f640 1 --2- 172.21.15.69:0/1783566867 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f82640744e0 0x7f82640769f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.918+0000 7f8288f5f640 1 --2- 172.21.15.69:0/1783566867 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82841644e0 0x7f82841eb290 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.918+0000 7f8288f5f640 1 --2- 172.21.15.69:0/1783566867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82840d1f30 0x7f8284150f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.918+0000 7f8288f5f640 1 --2- 172.21.15.69:0/1783566867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82840b8ca0 0x7f82840c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:41.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.918+0000 7f8288f5f640 1 -- 172.21.15.69:0/1783566867 >> 172.21.15.69:0/1783566867 conn(0x7f82840d3aa0 msgr2=0x7f82840b7630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:41.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.918+0000 7f8288f5f640 1 -- 172.21.15.69:0/1783566867 shutdown_connections 2024-04-25T11:00:41.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:41.918+0000 7f8288f5f640 1 -- 172.21.15.69:0/1783566867 wait complete. 2024-04-25T11:00:41.922 INFO:tasks.ceph.mds.c.smithi069.stderr: -556> 2024-04-25T11:00:36.070+0000 7f8cc1b21640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:00:41.934 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T11:00:41.973 INFO:tasks.ceph.mds.c.smithi069.stdout:starting mds.c at 2024-04-25T11:00:42.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.044+0000 7f1af0b72640 1 Processor -- start 2024-04-25T11:00:42.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.044+0000 7f1af0b72640 1 -- start start 2024-04-25T11:00:42.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.046+0000 7f1af0b72640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f1aec143660 0x7f1aec070290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:42.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.046+0000 7f1af0b72640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1aec06d5d0 0x7f1aec06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:42.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.046+0000 7f1af0b72640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1aec06df20 con 0x7f1aec0d04f0 2024-04-25T11:00:42.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.046+0000 7f1af0b72640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f1aec070c70 con 0x7f1aec143660 2024-04-25T11:00:42.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.046+0000 7f1af0b72640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f1aec0b4920 con 0x7f1aec06d5d0 2024-04-25T11:00:42.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.046+0000 7f1aea59c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f1aec143660 0x7f1aec070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:42.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.046+0000 7f1aea59c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f1aec143660 0x7f1aec070290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48088/0 (socket says 172.21.15.69:48088) 2024-04-25T11:00:42.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.046+0000 7f1aea59c640 1 -- 172.21.15.69:0/373942114 learned_addr learned my addr 172.21.15.69:0/373942114 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:42.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.046+0000 7f1ae9d9b640 1 --2- 172.21.15.69:0/373942114 >> v2:172.21.15.146:3300/0 conn(0x7f1aec06d5d0 0x7f1aec06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:42.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.046+0000 7f1aea59c640 1 -- 172.21.15.69:0/373942114 >> v2:172.21.15.146:3300/0 conn(0x7f1aec06d5d0 msgr2=0x7f1aec06d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:42.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.046+0000 7f1aea59c640 1 --2- 172.21.15.69:0/373942114 >> v2:172.21.15.146:3300/0 conn(0x7f1aec06d5d0 0x7f1aec06d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:42.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.046+0000 7f1aea59c640 1 -- 172.21.15.69:0/373942114 >> v1:172.21.15.146:6789/0 conn(0x7f1aec0d04f0 legacy=0x7f1aec0d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:42.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.046+0000 7f1aea59c640 1 -- 172.21.15.69:0/373942114 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aec0b4c00 con 0x7f1aec143660 2024-04-25T11:00:42.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.047+0000 7f1aea59c640 1 --2- 172.21.15.69:0/373942114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1aec143660 0x7f1aec070290 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1aec071210 tx=0x7f1ae002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3963b1329aa817b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:42.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.047+0000 7f1ae959a640 1 -- 172.21.15.69:0/373942114 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0037070 con 0x7f1aec143660 2024-04-25T11:00:42.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.047+0000 7f1ae959a640 1 -- 172.21.15.69:0/373942114 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ae0032440 con 0x7f1aec143660 2024-04-25T11:00:42.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.047+0000 7f1ae959a640 1 -- 172.21.15.69:0/373942114 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0032810 con 0x7f1aec143660 2024-04-25T11:00:42.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.048+0000 7f1af0b72640 1 -- 172.21.15.69:0/373942114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1aec143660 msgr2=0x7f1aec070290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:42.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.048+0000 7f1af0b72640 1 --2- 172.21.15.69:0/373942114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1aec143660 0x7f1aec070290 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1aec071210 tx=0x7f1ae002a1a0 comp rx=0 tx=0).stop 2024-04-25T11:00:42.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.048+0000 7f1af0b72640 1 -- 172.21.15.69:0/373942114 shutdown_connections 2024-04-25T11:00:42.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.048+0000 7f1af0b72640 1 --2- 172.21.15.69:0/373942114 >> v2:172.21.15.146:3300/0 conn(0x7f1aec06d5d0 0x7f1aec06d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:42.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.048+0000 7f1af0b72640 1 --2- 172.21.15.69:0/373942114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1aec143660 0x7f1aec070290 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:42.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.048+0000 7f1af0b72640 1 -- 172.21.15.69:0/373942114 >> 172.21.15.69:0/373942114 conn(0x7f1aec073120 msgr2=0x7f1aec073530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:42.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.048+0000 7f1af0b72640 1 -- 172.21.15.69:0/373942114 shutdown_connections 2024-04-25T11:00:42.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.048+0000 7f1af0b72640 1 -- 172.21.15.69:0/373942114 wait complete. 2024-04-25T11:00:42.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.050+0000 7f1af0b72640 1 Processor -- start 2024-04-25T11:00:42.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.050+0000 7f1af0b72640 1 -- start start 2024-04-25T11:00:42.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.051+0000 7f1af0b72640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1aec06d5d0 0x7f1aec1e8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:42.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.051+0000 7f1af0b72640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1aec0d04f0 0x7f1aec1e9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:42.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.051+0000 7f1af0b72640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1aec143660 0x7f1aec137430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:42.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.051+0000 7f1af0b72640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1aec0ce0e0 con 0x7f1aec143660 2024-04-25T11:00:42.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.051+0000 7f1af0b72640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1aec0b51c0 con 0x7f1aec0d04f0 2024-04-25T11:00:42.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.051+0000 7f1af0b72640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f1aec0ce260 con 0x7f1aec06d5d0 2024-04-25T11:00:42.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.051+0000 7f1aead9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1aec143660 0x7f1aec137430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:42.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.051+0000 7f1aead9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1aec143660 0x7f1aec137430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48096/0 (socket says 172.21.15.69:48096) 2024-04-25T11:00:42.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.051+0000 7f1aead9d640 1 -- 172.21.15.69:0/3813193349 learned_addr learned my addr 172.21.15.69:0/3813193349 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:42.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.051+0000 7f1aea59c640 1 --2- 172.21.15.69:0/3813193349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1aec06d5d0 0x7f1aec1e8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:42.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.051+0000 7f1ae9d9b640 1 --2- 172.21.15.69:0/3813193349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1aec0d04f0 0x7f1aec1e9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:42.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.051+0000 7f1aead9d640 1 -- 172.21.15.69:0/3813193349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1aec06d5d0 msgr2=0x7f1aec1e8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:42.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.051+0000 7f1aead9d640 1 --2- 172.21.15.69:0/3813193349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1aec06d5d0 0x7f1aec1e8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:42.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.051+0000 7f1aead9d640 1 -- 172.21.15.69:0/3813193349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1aec0d04f0 msgr2=0x7f1aec1e9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:42.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.051+0000 7f1aead9d640 1 --2- 172.21.15.69:0/3813193349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1aec0d04f0 0x7f1aec1e9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:42.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.051+0000 7f1aead9d640 1 -- 172.21.15.69:0/3813193349 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aec15cd20 con 0x7f1aec143660 2024-04-25T11:00:42.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.051+0000 7f1aea59c640 1 --2- 172.21.15.69:0/3813193349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1aec06d5d0 0x7f1aec1e8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:42.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.052+0000 7f1aead9d640 1 --2- 172.21.15.69:0/3813193349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1aec143660 0x7f1aec137430 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400b970 tx=0x7f1ae400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:42.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.052+0000 7f1adaffd640 1 -- 172.21.15.69:0/3813193349 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae401b070 con 0x7f1aec143660 2024-04-25T11:00:42.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.052+0000 7f1adaffd640 1 -- 172.21.15.69:0/3813193349 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ae400ed50 con 0x7f1aec143660 2024-04-25T11:00:42.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.052+0000 7f1af0b72640 1 -- 172.21.15.69:0/3813193349 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aec1e73e0 con 0x7f1aec143660 2024-04-25T11:00:42.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.052+0000 7f1adaffd640 1 -- 172.21.15.69:0/3813193349 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae4016680 con 0x7f1aec143660 2024-04-25T11:00:42.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.052+0000 7f1af0b72640 1 -- 172.21.15.69:0/3813193349 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aec1e79a0 con 0x7f1aec143660 2024-04-25T11:00:42.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.053+0000 7f1af0b72640 1 -- 172.21.15.69:0/3813193349 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aec050740 con 0x7f1aec143660 2024-04-25T11:00:42.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.054+0000 7f1adaffd640 1 -- 172.21.15.69:0/3813193349 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1ae401e070 con 0x7f1aec143660 2024-04-25T11:00:42.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.054+0000 7f1adaffd640 1 --2- 172.21.15.69:0/3813193349 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1acc0744b0 0x7f1acc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:42.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.054+0000 7f1aea59c640 1 --2- 172.21.15.69:0/3813193349 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1acc0744b0 0x7f1acc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:42.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.054+0000 7f1adaffd640 1 -- 172.21.15.69:0/3813193349 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5881+0+0 (secure 0 0 0) 0x7f1ae40971c0 con 0x7f1aec143660 2024-04-25T11:00:42.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.055+0000 7f1aea59c640 1 --2- 172.21.15.69:0/3813193349 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1acc0744b0 0x7f1acc0769c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0034000 tx=0x7f1ae002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:42.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.059+0000 7f1adaffd640 1 -- 172.21.15.69:0/3813193349 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1ae40618a0 con 0x7f1aec143660 2024-04-25T11:00:42.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.274+0000 7f1af0b72640 1 -- 172.21.15.69:0/3813193349 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f1aec06dd00 con 0x7f1aec143660 2024-04-25T11:00:42.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.918+0000 7f1adaffd640 1 -- 172.21.15.69:0/3813193349 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7f1ae40611a0 con 0x7f1aec143660 2024-04-25T11:00:42.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.920+0000 7f1af0b72640 1 -- 172.21.15.69:0/3813193349 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1acc0744b0 msgr2=0x7f1acc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:42.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.920+0000 7f1af0b72640 1 --2- 172.21.15.69:0/3813193349 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1acc0744b0 0x7f1acc0769c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0034000 tx=0x7f1ae002e000 comp rx=0 tx=0).stop 2024-04-25T11:00:42.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.920+0000 7f1af0b72640 1 -- 172.21.15.69:0/3813193349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1aec143660 msgr2=0x7f1aec137430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:42.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.920+0000 7f1af0b72640 1 --2- 172.21.15.69:0/3813193349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1aec143660 0x7f1aec137430 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400b970 tx=0x7f1ae400be30 comp rx=0 tx=0).stop 2024-04-25T11:00:42.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.920+0000 7f1af0b72640 1 -- 172.21.15.69:0/3813193349 shutdown_connections 2024-04-25T11:00:42.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.920+0000 7f1af0b72640 1 --2- 172.21.15.69:0/3813193349 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f1acc0744b0 0x7f1acc0769c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:42.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.920+0000 7f1af0b72640 1 --2- 172.21.15.69:0/3813193349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1aec143660 0x7f1aec137430 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:42.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.920+0000 7f1af0b72640 1 --2- 172.21.15.69:0/3813193349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1aec0d04f0 0x7f1aec1e9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:42.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.920+0000 7f1af0b72640 1 --2- 172.21.15.69:0/3813193349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1aec06d5d0 0x7f1aec1e8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:42.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.920+0000 7f1af0b72640 1 -- 172.21.15.69:0/3813193349 >> 172.21.15.69:0/3813193349 conn(0x7f1aec073120 msgr2=0x7f1aec142440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:42.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.920+0000 7f1af0b72640 1 -- 172.21.15.69:0/3813193349 shutdown_connections 2024-04-25T11:00:42.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:42.920+0000 7f1af0b72640 1 -- 172.21.15.69:0/3813193349 wait complete. 2024-04-25T11:00:42.934 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T11:00:43.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.090+0000 7ff0b6f9d640 1 Processor -- start 2024-04-25T11:00:43.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.090+0000 7ff0b6f9d640 1 -- start start 2024-04-25T11:00:43.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.091+0000 7ff0b6f9d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0b00d3e00 0x7ff0b01601a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:43.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.091+0000 7ff0b6f9d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff0b00d4660 0x7ff0b00b7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:43.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.091+0000 7ff0b6f9d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0b01606e0 con 0x7ff0b00b7e90 2024-04-25T11:00:43.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.091+0000 7ff0b6f9d640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7ff0b00b82a0 con 0x7ff0b00d4660 2024-04-25T11:00:43.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.091+0000 7ff0b6f9d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff0b00d4c20 con 0x7ff0b00d3e00 2024-04-25T11:00:43.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.091+0000 7ff0b5f9b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0b00d3e00 0x7ff0b01601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:43.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.091+0000 7ff0b679c640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7ff0b00b7e90 0x7ff0b006daf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.69:44362/0 (socket says 172.21.15.69:44362) 2024-04-25T11:00:43.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.091+0000 7ff0b579a640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff0b00d4660 0x7ff0b00b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:43.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.091+0000 7ff0b5f9b640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0b00d3e00 0x7ff0b01601a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.69:53162/0 (socket says 172.21.15.69:53162) 2024-04-25T11:00:43.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.091+0000 7ff0b679c640 1 -- 172.21.15.69:0/677626107 learned_addr learned my addr 172.21.15.69:0/677626107 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:43.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.091+0000 7ff0b5f9b640 1 -- 172.21.15.69:0/677626107 >> v2:172.21.15.146:3300/0 conn(0x7ff0b00d4660 msgr2=0x7ff0b00b7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:43.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.091+0000 7ff0b5f9b640 1 --2- 172.21.15.69:0/677626107 >> v2:172.21.15.146:3300/0 conn(0x7ff0b00d4660 0x7ff0b00b7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:43.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.091+0000 7ff0b5f9b640 1 -- 172.21.15.69:0/677626107 >> v1:172.21.15.146:6789/0 conn(0x7ff0b00b7e90 legacy=0x7ff0b006daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:43.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.091+0000 7ff0b5f9b640 1 -- 172.21.15.69:0/677626107 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b00d4f00 con 0x7ff0b00d3e00 2024-04-25T11:00:43.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.091+0000 7ff0b579a640 1 --2- 172.21.15.69:0/677626107 >> v2:172.21.15.146:3300/0 conn(0x7ff0b00d4660 0x7ff0b00b7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:00:43.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.092+0000 7ff0b5f9b640 1 --2- 172.21.15.69:0/677626107 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0b00d3e00 0x7ff0b01601a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac009de0 tx=0x7ff0ac02a630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c767e48ece0059b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:43.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.092+0000 7ff0b4f99640 1 -- 172.21.15.69:0/677626107 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac007740 con 0x7ff0b00d3e00 2024-04-25T11:00:43.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.093+0000 7ff0b4f99640 1 -- 172.21.15.69:0/677626107 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0ac02ae50 con 0x7ff0b00d3e00 2024-04-25T11:00:43.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.093+0000 7ff0b4f99640 1 -- 172.21.15.69:0/677626107 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac007b00 con 0x7ff0b00d3e00 2024-04-25T11:00:43.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.093+0000 7ff0b6f9d640 1 -- 172.21.15.69:0/677626107 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0b00d3e00 msgr2=0x7ff0b01601a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:43.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.093+0000 7ff0b6f9d640 1 --2- 172.21.15.69:0/677626107 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0b00d3e00 0x7ff0b01601a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac009de0 tx=0x7ff0ac02a630 comp rx=0 tx=0).stop 2024-04-25T11:00:43.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.093+0000 7ff0b6f9d640 1 -- 172.21.15.69:0/677626107 shutdown_connections 2024-04-25T11:00:43.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.093+0000 7ff0b6f9d640 1 --2- 172.21.15.69:0/677626107 >> v2:172.21.15.146:3300/0 conn(0x7ff0b00d4660 0x7ff0b00b7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:43.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.093+0000 7ff0b6f9d640 1 --2- 172.21.15.69:0/677626107 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0b00d3e00 0x7ff0b01601a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:43.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.093+0000 7ff0b6f9d640 1 -- 172.21.15.69:0/677626107 >> 172.21.15.69:0/677626107 conn(0x7ff0b00d29f0 msgr2=0x7ff0b00d2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:43.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.093+0000 7ff0b6f9d640 1 -- 172.21.15.69:0/677626107 shutdown_connections 2024-04-25T11:00:43.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.093+0000 7ff0b6f9d640 1 -- 172.21.15.69:0/677626107 wait complete. 2024-04-25T11:00:43.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.095+0000 7ff0b6f9d640 1 Processor -- start 2024-04-25T11:00:43.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.095+0000 7ff0b6f9d640 1 -- start start 2024-04-25T11:00:43.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.096+0000 7ff0b6f9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0b00b7e90 0x7ff0b0150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:43.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.096+0000 7ff0b6f9d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0b00d3e00 0x7ff0b00cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:43.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.096+0000 7ff0b6f9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0b00d4660 0x7ff0b00cc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:43.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.096+0000 7ff0b6f9d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b00cc9e0 con 0x7ff0b00b7e90 2024-04-25T11:00:43.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.096+0000 7ff0b6f9d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b00d54c0 con 0x7ff0b00d4660 2024-04-25T11:00:43.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.096+0000 7ff0b6f9d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff0b00b91d0 con 0x7ff0b00d3e00 2024-04-25T11:00:43.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.097+0000 7ff0b5f9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0b00b7e90 0x7ff0b0150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:43.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.097+0000 7ff0b5f9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0b00b7e90 0x7ff0b0150580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48110/0 (socket says 172.21.15.69:48110) 2024-04-25T11:00:43.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.097+0000 7ff0b5f9b640 1 -- 172.21.15.69:0/706579140 learned_addr learned my addr 172.21.15.69:0/706579140 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:43.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.097+0000 7ff0b679c640 1 --2- 172.21.15.69:0/706579140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0b00d4660 0x7ff0b00cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:43.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.097+0000 7ff0b579a640 1 --2- 172.21.15.69:0/706579140 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0b00d3e00 0x7ff0b00cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:43.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.097+0000 7ff0b5f9b640 1 -- 172.21.15.69:0/706579140 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0b00d3e00 msgr2=0x7ff0b00cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:43.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.097+0000 7ff0b5f9b640 1 --2- 172.21.15.69:0/706579140 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0b00d3e00 0x7ff0b00cbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:43.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.097+0000 7ff0b5f9b640 1 -- 172.21.15.69:0/706579140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0b00d4660 msgr2=0x7ff0b00cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:43.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.097+0000 7ff0b5f9b640 1 --2- 172.21.15.69:0/706579140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0b00d4660 0x7ff0b00cc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:43.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.097+0000 7ff0b5f9b640 1 -- 172.21.15.69:0/706579140 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b0154d90 con 0x7ff0b00b7e90 2024-04-25T11:00:43.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.097+0000 7ff0b5f9b640 1 --2- 172.21.15.69:0/706579140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0b00b7e90 0x7ff0b0150580 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac009f00 tx=0x7ff0ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:43.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.097+0000 7ff0a67fc640 1 -- 172.21.15.69:0/706579140 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac03f070 con 0x7ff0b00b7e90 2024-04-25T11:00:43.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.097+0000 7ff0a67fc640 1 -- 172.21.15.69:0/706579140 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0ac03b820 con 0x7ff0b00b7e90 2024-04-25T11:00:43.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.097+0000 7ff0b6f9d640 1 -- 172.21.15.69:0/706579140 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0b0155070 con 0x7ff0b00b7e90 2024-04-25T11:00:43.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.097+0000 7ff0a67fc640 1 -- 172.21.15.69:0/706579140 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac03bbd0 con 0x7ff0b00b7e90 2024-04-25T11:00:43.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.097+0000 7ff0b6f9d640 1 -- 172.21.15.69:0/706579140 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0b015c6f0 con 0x7ff0b00b7e90 2024-04-25T11:00:43.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.098+0000 7ff0b6f9d640 1 -- 172.21.15.69:0/706579140 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0b015c950 con 0x7ff0b00b7e90 2024-04-25T11:00:43.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.099+0000 7ff0a67fc640 1 -- 172.21.15.69:0/706579140 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff0ac044460 con 0x7ff0b00b7e90 2024-04-25T11:00:43.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.099+0000 7ff0a67fc640 1 --2- 172.21.15.69:0/706579140 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff0940744e0 0x7ff0940769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:43.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.099+0000 7ff0a67fc640 1 -- 172.21.15.69:0/706579140 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5881+0+0 (secure 0 0 0) 0x7ff0ac0b6780 con 0x7ff0b00b7e90 2024-04-25T11:00:43.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.100+0000 7ff0b579a640 1 --2- 172.21.15.69:0/706579140 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff0940744e0 0x7ff0940769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:43.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.100+0000 7ff0b579a640 1 --2- 172.21.15.69:0/706579140 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff0940744e0 0x7ff0940769f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff0b015cf10 tx=0x7ff0a0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:43.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.104+0000 7ff0a67fc640 1 -- 172.21.15.69:0/706579140 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff0ac080810 con 0x7ff0b00b7e90 2024-04-25T11:00:43.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:43.278+0000 7ff0b6f9d640 1 -- 172.21.15.69:0/706579140 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff0b0076c60 con 0x7ff0b00b7e90 2024-04-25T11:00:44.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.130+0000 7ff0a67fc640 1 -- 172.21.15.69:0/706579140 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v34) ==== 181+0+0 (secure 0 0 0) 0x7ff0ac084b10 con 0x7ff0b00b7e90 2024-04-25T11:00:44.130 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_metadata' removed 2024-04-25T11:00:44.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.134+0000 7ff0b6f9d640 1 -- 172.21.15.69:0/706579140 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff0940744e0 msgr2=0x7ff0940769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:44.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.134+0000 7ff0b6f9d640 1 --2- 172.21.15.69:0/706579140 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff0940744e0 0x7ff0940769f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff0b015cf10 tx=0x7ff0a0009190 comp rx=0 tx=0).stop 2024-04-25T11:00:44.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.134+0000 7ff0b6f9d640 1 -- 172.21.15.69:0/706579140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0b00b7e90 msgr2=0x7ff0b0150580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:44.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.134+0000 7ff0b6f9d640 1 --2- 172.21.15.69:0/706579140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0b00b7e90 0x7ff0b0150580 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac009f00 tx=0x7ff0ac004a40 comp rx=0 tx=0).stop 2024-04-25T11:00:44.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.135+0000 7ff0b6f9d640 1 -- 172.21.15.69:0/706579140 shutdown_connections 2024-04-25T11:00:44.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.135+0000 7ff0b6f9d640 1 --2- 172.21.15.69:0/706579140 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff0940744e0 0x7ff0940769f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:44.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.135+0000 7ff0b6f9d640 1 --2- 172.21.15.69:0/706579140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0b00d4660 0x7ff0b00cc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:44.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.135+0000 7ff0b6f9d640 1 --2- 172.21.15.69:0/706579140 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff0b00d3e00 0x7ff0b00cbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:44.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.135+0000 7ff0b6f9d640 1 --2- 172.21.15.69:0/706579140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0b00b7e90 0x7ff0b0150580 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:44.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.135+0000 7ff0b6f9d640 1 -- 172.21.15.69:0/706579140 >> 172.21.15.69:0/706579140 conn(0x7ff0b00d29f0 msgr2=0x7ff0b00d6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:44.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.135+0000 7ff0b6f9d640 1 -- 172.21.15.69:0/706579140 shutdown_connections 2024-04-25T11:00:44.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.135+0000 7ff0b6f9d640 1 -- 172.21.15.69:0/706579140 wait complete. 2024-04-25T11:00:44.149 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T11:00:44.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.269+0000 7fb68e669640 1 Processor -- start 2024-04-25T11:00:44.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.269+0000 7fb68e669640 1 -- start start 2024-04-25T11:00:44.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.270+0000 7fb68e669640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb688161710 0x7fb68815e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:44.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.270+0000 7fb68e669640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb68806c5c0 0x7fb68806c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:44.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.270+0000 7fb68e669640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6881609b0 con 0x7fb688163fa0 2024-04-25T11:00:44.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.270+0000 7fb68e669640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fb68806cf10 con 0x7fb68806c5c0 2024-04-25T11:00:44.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.270+0000 7fb68e669640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb688164cd0 con 0x7fb688161710 2024-04-25T11:00:44.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.270+0000 7fb6877fe640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb68806c5c0 0x7fb68806c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:44.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.270+0000 7fb6877fe640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb68806c5c0 0x7fb68806c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48124/0 (socket says 172.21.15.69:48124) 2024-04-25T11:00:44.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.270+0000 7fb6877fe640 1 -- 172.21.15.69:0/2570469981 learned_addr learned my addr 172.21.15.69:0/2570469981 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:44.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.271+0000 7fb687fff640 1 --2- 172.21.15.69:0/2570469981 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb688161710 0x7fb68815e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:44.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.271+0000 7fb6877fe640 1 -- 172.21.15.69:0/2570469981 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb688161710 msgr2=0x7fb68815e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:44.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.271+0000 7fb6877fe640 1 --2- 172.21.15.69:0/2570469981 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb688161710 0x7fb68815e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:44.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.271+0000 7fb6877fe640 1 -- 172.21.15.69:0/2570469981 >> v1:172.21.15.146:6789/0 conn(0x7fb688163fa0 legacy=0x7fb6881643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:44.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.271+0000 7fb6877fe640 1 -- 172.21.15.69:0/2570469981 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb688160c30 con 0x7fb68806c5c0 2024-04-25T11:00:44.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.271+0000 7fb6877fe640 1 --2- 172.21.15.69:0/2570469981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb68806c5c0 0x7fb68806c9d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb67c009870 tx=0x7fb67c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de89fbf8efe825e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:44.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.271+0000 7fb686ffd640 1 -- 172.21.15.69:0/2570469981 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67c037070 con 0x7fb68806c5c0 2024-04-25T11:00:44.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.272+0000 7fb686ffd640 1 -- 172.21.15.69:0/2570469981 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb67c032440 con 0x7fb68806c5c0 2024-04-25T11:00:44.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.272+0000 7fb686ffd640 1 -- 172.21.15.69:0/2570469981 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67c032810 con 0x7fb68806c5c0 2024-04-25T11:00:44.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.272+0000 7fb68e669640 1 -- 172.21.15.69:0/2570469981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb68806c5c0 msgr2=0x7fb68806c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:44.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.272+0000 7fb68e669640 1 --2- 172.21.15.69:0/2570469981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb68806c5c0 0x7fb68806c9d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb67c009870 tx=0x7fb67c02a180 comp rx=0 tx=0).stop 2024-04-25T11:00:44.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.272+0000 7fb68e669640 1 -- 172.21.15.69:0/2570469981 shutdown_connections 2024-04-25T11:00:44.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.272+0000 7fb68e669640 1 --2- 172.21.15.69:0/2570469981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb68806c5c0 0x7fb68806c9d0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:44.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.272+0000 7fb68e669640 1 --2- 172.21.15.69:0/2570469981 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb688161710 0x7fb68815e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:44.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.272+0000 7fb68e669640 1 -- 172.21.15.69:0/2570469981 >> 172.21.15.69:0/2570469981 conn(0x7fb6880d5d90 msgr2=0x7fb6880d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:44.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.272+0000 7fb68e669640 1 -- 172.21.15.69:0/2570469981 shutdown_connections 2024-04-25T11:00:44.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.272+0000 7fb68e669640 1 -- 172.21.15.69:0/2570469981 wait complete. 2024-04-25T11:00:44.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.274+0000 7fb68e669640 1 Processor -- start 2024-04-25T11:00:44.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.274+0000 7fb68e669640 1 -- start start 2024-04-25T11:00:44.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.275+0000 7fb68e669640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb68806c5c0 0x7fb688152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:44.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.275+0000 7fb68e669640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb688161710 0x7fb6881ec040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:44.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.275+0000 7fb68e669640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb688163fa0 0x7fb68814f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:44.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.276+0000 7fb68e669640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb6880d49f0 con 0x7fb68806c5c0 2024-04-25T11:00:44.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.276+0000 7fb68e669640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb6881611f0 con 0x7fb688163fa0 2024-04-25T11:00:44.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.276+0000 7fb68e669640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb6880d4b70 con 0x7fb688161710 2024-04-25T11:00:44.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.276+0000 7fb687fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb68806c5c0 0x7fb688152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:44.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.276+0000 7fb687fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb68806c5c0 0x7fb688152d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48134/0 (socket says 172.21.15.69:48134) 2024-04-25T11:00:44.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.276+0000 7fb687fff640 1 -- 172.21.15.69:0/1974289039 learned_addr learned my addr 172.21.15.69:0/1974289039 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:44.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.276+0000 7fb6877fe640 1 --2- 172.21.15.69:0/1974289039 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb688161710 0x7fb6881ec040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:44.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.276+0000 7fb687fff640 1 -- 172.21.15.69:0/1974289039 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb688161710 msgr2=0x7fb6881ec040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:44.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.276+0000 7fb68cc06640 1 --2- 172.21.15.69:0/1974289039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb688163fa0 0x7fb68814f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:44.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.276+0000 7fb687fff640 1 --2- 172.21.15.69:0/1974289039 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb688161710 0x7fb6881ec040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:44.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.276+0000 7fb687fff640 1 -- 172.21.15.69:0/1974289039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb688163fa0 msgr2=0x7fb68814f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:44.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.276+0000 7fb687fff640 1 --2- 172.21.15.69:0/1974289039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb688163fa0 0x7fb68814f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:44.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.276+0000 7fb687fff640 1 -- 172.21.15.69:0/1974289039 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6880ccdd0 con 0x7fb68806c5c0 2024-04-25T11:00:44.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.276+0000 7fb68cc06640 1 --2- 172.21.15.69:0/1974289039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb688163fa0 0x7fb68814f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:00:44.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.276+0000 7fb687fff640 1 --2- 172.21.15.69:0/1974289039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb68806c5c0 0x7fb688152d30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb67800d5d0 tx=0x7fb67800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:44.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.276+0000 7fb684ff9640 1 -- 172.21.15.69:0/1974289039 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb678004240 con 0x7fb68806c5c0 2024-04-25T11:00:44.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.276+0000 7fb684ff9640 1 -- 172.21.15.69:0/1974289039 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6780043e0 con 0x7fb68806c5c0 2024-04-25T11:00:44.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.276+0000 7fb68e669640 1 -- 172.21.15.69:0/1974289039 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6880cd0b0 con 0x7fb68806c5c0 2024-04-25T11:00:44.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.276+0000 7fb684ff9640 1 -- 172.21.15.69:0/1974289039 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb678003830 con 0x7fb68806c5c0 2024-04-25T11:00:44.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.276+0000 7fb68e669640 1 -- 172.21.15.69:0/1974289039 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6880cd670 con 0x7fb68806c5c0 2024-04-25T11:00:44.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.277+0000 7fb68e669640 1 -- 172.21.15.69:0/1974289039 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb688050740 con 0x7fb68806c5c0 2024-04-25T11:00:44.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.278+0000 7fb684ff9640 1 -- 172.21.15.69:0/1974289039 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb6780039d0 con 0x7fb68806c5c0 2024-04-25T11:00:44.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.278+0000 7fb684ff9640 1 --2- 172.21.15.69:0/1974289039 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb6740744e0 0x7fb6740769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:44.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.278+0000 7fb684ff9640 1 -- 172.21.15.69:0/1974289039 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5443+0+0 (secure 0 0 0) 0x7fb678095440 con 0x7fb68806c5c0 2024-04-25T11:00:44.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.278+0000 7fb6877fe640 1 --2- 172.21.15.69:0/1974289039 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb6740744e0 0x7fb6740769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:44.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.279+0000 7fb6877fe640 1 --2- 172.21.15.69:0/1974289039 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb6740744e0 0x7fb6740769f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb67c034000 tx=0x7fb67c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:44.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.283+0000 7fb684ff9640 1 -- 172.21.15.69:0/1974289039 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb67805fcd0 con 0x7fb68806c5c0 2024-04-25T11:00:44.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:44.465+0000 7fb68e669640 1 -- 172.21.15.69:0/1974289039 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb6881637f0 con 0x7fb68806c5c0 2024-04-25T11:00:45.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.134+0000 7fb684ff9640 1 -- 172.21.15.69:0/1974289039 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v35) ==== 169+0+0 (secure 0 0 0) 0x7fb67805f5d0 con 0x7fb68806c5c0 2024-04-25T11:00:45.135 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_data' removed 2024-04-25T11:00:45.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.137+0000 7fb68e669640 1 -- 172.21.15.69:0/1974289039 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb6740744e0 msgr2=0x7fb6740769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:45.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.137+0000 7fb68e669640 1 --2- 172.21.15.69:0/1974289039 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb6740744e0 0x7fb6740769f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb67c034000 tx=0x7fb67c02e000 comp rx=0 tx=0).stop 2024-04-25T11:00:45.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.137+0000 7fb68e669640 1 -- 172.21.15.69:0/1974289039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb68806c5c0 msgr2=0x7fb688152d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:45.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.137+0000 7fb68e669640 1 --2- 172.21.15.69:0/1974289039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb68806c5c0 0x7fb688152d30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb67800d5d0 tx=0x7fb67800da90 comp rx=0 tx=0).stop 2024-04-25T11:00:45.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.138+0000 7fb68e669640 1 -- 172.21.15.69:0/1974289039 shutdown_connections 2024-04-25T11:00:45.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.138+0000 7fb68e669640 1 --2- 172.21.15.69:0/1974289039 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb6740744e0 0x7fb6740769f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:45.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.138+0000 7fb68e669640 1 --2- 172.21.15.69:0/1974289039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb688163fa0 0x7fb68814f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:45.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.138+0000 7fb68e669640 1 --2- 172.21.15.69:0/1974289039 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb688161710 0x7fb6881ec040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:45.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.138+0000 7fb68e669640 1 --2- 172.21.15.69:0/1974289039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb68806c5c0 0x7fb688152d30 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:45.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.138+0000 7fb68e669640 1 -- 172.21.15.69:0/1974289039 >> 172.21.15.69:0/1974289039 conn(0x7fb6880d5d90 msgr2=0x7fb68815e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:45.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.139+0000 7fb68e669640 1 -- 172.21.15.69:0/1974289039 shutdown_connections 2024-04-25T11:00:45.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.139+0000 7fb68e669640 1 -- 172.21.15.69:0/1974289039 wait complete. 2024-04-25T11:00:45.154 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T11:00:45.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.265+0000 7f786319d640 1 Processor -- start 2024-04-25T11:00:45.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.265+0000 7f786319d640 1 -- start start 2024-04-25T11:00:45.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.266+0000 7f786319d640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f785c0d0190 0x7f785c0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:45.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.266+0000 7f786319d640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f785c0b7ef0 0x7f785c0b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:45.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.266+0000 7f786319d640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f785c0b3f30 con 0x7f785c0d1f90 2024-04-25T11:00:45.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.266+0000 7f786319d640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f785c0b23e0 con 0x7f785c0d0190 2024-04-25T11:00:45.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.266+0000 7f786319d640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f785c0b4210 con 0x7f785c0b7ef0 2024-04-25T11:00:45.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.266+0000 7f7860f39640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f785c0d0190 0x7f785c0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:45.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.266+0000 7f785bfff640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f785c0d1f90 0x7f785c0b77e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:39522/0 (socket says 172.21.15.69:39522) 2024-04-25T11:00:45.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.266+0000 7f7860f39640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f785c0d0190 0x7f785c0d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48140/0 (socket says 172.21.15.69:48140) 2024-04-25T11:00:45.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.266+0000 7f785bfff640 1 -- 172.21.15.69:0/2385534260 learned_addr learned my addr 172.21.15.69:0/2385534260 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:45.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.267+0000 7f7860f39640 1 -- 172.21.15.69:0/2385534260 >> v2:172.21.15.146:3300/0 conn(0x7f785c0b7ef0 msgr2=0x7f785c0b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:45.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.267+0000 7f7860f39640 1 --2- 172.21.15.69:0/2385534260 >> v2:172.21.15.146:3300/0 conn(0x7f785c0b7ef0 0x7f785c0b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:45.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.267+0000 7f7860f39640 1 -- 172.21.15.69:0/2385534260 >> v1:172.21.15.69:6789/0 conn(0x7f785c0d1f90 legacy=0x7f785c0b77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:45.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.267+0000 7f7860f39640 1 -- 172.21.15.69:0/2385534260 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f785c0b2630 con 0x7f785c0d0190 2024-04-25T11:00:45.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.267+0000 7f7860f39640 1 --2- 172.21.15.69:0/2385534260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f785c0d0190 0x7f785c0d1a50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f785c09c510 tx=0x7f785002a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a55c92cc2e5e7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:45.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.268+0000 7f785b7fe640 1 -- 172.21.15.69:0/2385534260 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850037070 con 0x7f785c0d0190 2024-04-25T11:00:45.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.268+0000 7f785b7fe640 1 -- 172.21.15.69:0/2385534260 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f785002add0 con 0x7f785c0d0190 2024-04-25T11:00:45.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.268+0000 7f785b7fe640 1 -- 172.21.15.69:0/2385534260 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850032660 con 0x7f785c0d0190 2024-04-25T11:00:45.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.268+0000 7f786319d640 1 -- 172.21.15.69:0/2385534260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f785c0d0190 msgr2=0x7f785c0d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:45.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.268+0000 7f786319d640 1 --2- 172.21.15.69:0/2385534260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f785c0d0190 0x7f785c0d1a50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f785c09c510 tx=0x7f785002a080 comp rx=0 tx=0).stop 2024-04-25T11:00:45.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.268+0000 7f786319d640 1 -- 172.21.15.69:0/2385534260 shutdown_connections 2024-04-25T11:00:45.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.268+0000 7f786319d640 1 --2- 172.21.15.69:0/2385534260 >> v2:172.21.15.146:3300/0 conn(0x7f785c0b7ef0 0x7f785c0b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:45.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.268+0000 7f786319d640 1 --2- 172.21.15.69:0/2385534260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f785c0d0190 0x7f785c0d1a50 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:45.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.268+0000 7f786319d640 1 -- 172.21.15.69:0/2385534260 >> 172.21.15.69:0/2385534260 conn(0x7f785c09b7d0 msgr2=0x7f785c09bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:45.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.268+0000 7f786319d640 1 -- 172.21.15.69:0/2385534260 shutdown_connections 2024-04-25T11:00:45.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.268+0000 7f786319d640 1 -- 172.21.15.69:0/2385534260 wait complete. 2024-04-25T11:00:45.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.270+0000 7f786319d640 1 Processor -- start 2024-04-25T11:00:45.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.270+0000 7f786319d640 1 -- start start 2024-04-25T11:00:45.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.271+0000 7f786319d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f785c0d0190 0x7f785c0c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:45.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.271+0000 7f786319d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f785c0d1f90 0x7f785c0c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:45.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.272+0000 7f786319d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f785c153120 0x7f785c0cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:45.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.272+0000 7f786319d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f785c06ee50 con 0x7f785c0d1f90 2024-04-25T11:00:45.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.272+0000 7f786319d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f785c06ecd0 con 0x7f785c153120 2024-04-25T11:00:45.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.272+0000 7f786319d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f785c06efd0 con 0x7f785c0d0190 2024-04-25T11:00:45.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.272+0000 7f785bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f785c0d1f90 0x7f785c0c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:45.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.272+0000 7f785bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f785c0d1f90 0x7f785c0c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48144/0 (socket says 172.21.15.69:48144) 2024-04-25T11:00:45.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.272+0000 7f785bfff640 1 -- 172.21.15.69:0/1815381565 learned_addr learned my addr 172.21.15.69:0/1815381565 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:45.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.272+0000 7f7860f39640 1 --2- 172.21.15.69:0/1815381565 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f785c0d0190 0x7f785c0c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:45.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.272+0000 7f785bfff640 1 -- 172.21.15.69:0/1815381565 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f785c0d0190 msgr2=0x7f785c0c6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:45.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.272+0000 7f785bfff640 1 --2- 172.21.15.69:0/1815381565 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f785c0d0190 0x7f785c0c6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:45.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.272+0000 7f785bfff640 1 -- 172.21.15.69:0/1815381565 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f785c153120 msgr2=0x7f785c0cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:45.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.272+0000 7f785bfff640 1 --2- 172.21.15.69:0/1815381565 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f785c153120 0x7f785c0cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:45.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.272+0000 7f785bfff640 1 -- 172.21.15.69:0/1815381565 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f785c0c7810 con 0x7f785c0d1f90 2024-04-25T11:00:45.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.272+0000 7f785bfff640 1 --2- 172.21.15.69:0/1815381565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f785c0d1f90 0x7f785c0c86b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f784800c3f0 tx=0x7f784800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:45.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.272+0000 7f78597fa640 1 -- 172.21.15.69:0/1815381565 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848017070 con 0x7f785c0d1f90 2024-04-25T11:00:45.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.272+0000 7f786319d640 1 -- 172.21.15.69:0/1815381565 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f785c0c7af0 con 0x7f785c0d1f90 2024-04-25T11:00:45.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.272+0000 7f78597fa640 1 -- 172.21.15.69:0/1815381565 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7848005ce0 con 0x7f785c0d1f90 2024-04-25T11:00:45.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.272+0000 7f78597fa640 1 -- 172.21.15.69:0/1815381565 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78480072f0 con 0x7f785c0d1f90 2024-04-25T11:00:45.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.272+0000 7f786319d640 1 -- 172.21.15.69:0/1815381565 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f785c0c9eb0 con 0x7f785c0d1f90 2024-04-25T11:00:45.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.273+0000 7f786319d640 1 -- 172.21.15.69:0/1815381565 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f785c0d0e60 con 0x7f785c0d1f90 2024-04-25T11:00:45.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.274+0000 7f78597fa640 1 -- 172.21.15.69:0/1815381565 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f784801a070 con 0x7f785c0d1f90 2024-04-25T11:00:45.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.275+0000 7f78597fa640 1 --2- 172.21.15.69:0/1815381565 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f78400744e0 0x7f78400769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:45.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.275+0000 7f7860f39640 1 --2- 172.21.15.69:0/1815381565 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f78400744e0 0x7f78400769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:45.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.275+0000 7f78597fa640 1 -- 172.21.15.69:0/1815381565 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7f7848094380 con 0x7f785c0d1f90 2024-04-25T11:00:45.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.275+0000 7f7860f39640 1 --2- 172.21.15.69:0/1815381565 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f78400744e0 0x7f78400769f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7850034000 tx=0x7f785002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:45.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.279+0000 7f78597fa640 1 -- 172.21.15.69:0/1815381565 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f785c0d0e60 con 0x7f785c0d1f90 2024-04-25T11:00:45.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.464+0000 7f786319d640 1 -- 172.21.15.69:0/1815381565 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f785c050740 con 0x7f785c0d1f90 2024-04-25T11:00:45.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.467+0000 7f78597fa640 1 -- 172.21.15.69:0/1815381565 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f784805ed30 con 0x7f785c0d1f90 2024-04-25T11:00:45.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.469+0000 7f786319d640 1 -- 172.21.15.69:0/1815381565 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f78400744e0 msgr2=0x7f78400769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:45.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.469+0000 7f786319d640 1 --2- 172.21.15.69:0/1815381565 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f78400744e0 0x7f78400769f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7850034000 tx=0x7f785002e000 comp rx=0 tx=0).stop 2024-04-25T11:00:45.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.469+0000 7f786319d640 1 -- 172.21.15.69:0/1815381565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f785c0d1f90 msgr2=0x7f785c0c86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:45.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.469+0000 7f786319d640 1 --2- 172.21.15.69:0/1815381565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f785c0d1f90 0x7f785c0c86b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f784800c3f0 tx=0x7f784800c8b0 comp rx=0 tx=0).stop 2024-04-25T11:00:45.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.469+0000 7f786319d640 1 -- 172.21.15.69:0/1815381565 shutdown_connections 2024-04-25T11:00:45.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.469+0000 7f786319d640 1 --2- 172.21.15.69:0/1815381565 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f78400744e0 0x7f78400769f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:45.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.469+0000 7f786319d640 1 --2- 172.21.15.69:0/1815381565 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f785c153120 0x7f785c0cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:45.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.469+0000 7f786319d640 1 --2- 172.21.15.69:0/1815381565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f785c0d1f90 0x7f785c0c86b0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:45.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.469+0000 7f786319d640 1 --2- 172.21.15.69:0/1815381565 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f785c0d0190 0x7f785c0c6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:45.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.469+0000 7f786319d640 1 -- 172.21.15.69:0/1815381565 >> 172.21.15.69:0/1815381565 conn(0x7f785c09b7d0 msgr2=0x7f785c0b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:45.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.469+0000 7f786319d640 1 -- 172.21.15.69:0/1815381565 shutdown_connections 2024-04-25T11:00:45.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.469+0000 7f786319d640 1 -- 172.21.15.69:0/1815381565 wait complete. 2024-04-25T11:00:45.481 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_cap_flush.TestCapFlush.test_replay_create' 2024-04-25T11:00:45.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.587+0000 7f02afef2640 1 Processor -- start 2024-04-25T11:00:45.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.587+0000 7f02afef2640 1 -- start start 2024-04-25T11:00:45.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.588+0000 7f02afef2640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f02a80b8ca0 0x7f02a8163fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:45.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.588+0000 7f02afef2640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02a80ce060 0x7f02a80ce470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:45.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.588+0000 7f02afef2640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02a806ed60 con 0x7f02a81644e0 2024-04-25T11:00:45.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.588+0000 7f02afef2640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f02a80d35a0 con 0x7f02a80b8ca0 2024-04-25T11:00:45.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.588+0000 7f02afef2640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f02a806e8d0 con 0x7f02a80ce060 2024-04-25T11:00:45.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.588+0000 7f02adc8e640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f02a80b8ca0 0x7f02a8163fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:45.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.588+0000 7f02ad48d640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f02a81644e0 0x7f02a81648f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:54390/0 (socket says 172.21.15.69:54390) 2024-04-25T11:00:45.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.588+0000 7f02adc8e640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f02a80b8ca0 0x7f02a8163fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47762/0 (socket says 172.21.15.69:47762) 2024-04-25T11:00:45.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.588+0000 7f02ad48d640 1 -- 172.21.15.69:0/3300361977 learned_addr learned my addr 172.21.15.69:0/3300361977 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:45.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.589+0000 7f02adc8e640 1 -- 172.21.15.69:0/3300361977 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02a80ce060 msgr2=0x7f02a80ce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:45.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.589+0000 7f02ae48f640 1 --2- 172.21.15.69:0/3300361977 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02a80ce060 0x7f02a80ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:45.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.589+0000 7f02adc8e640 1 --2- 172.21.15.69:0/3300361977 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02a80ce060 0x7f02a80ce470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:45.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.589+0000 7f02adc8e640 1 -- 172.21.15.69:0/3300361977 >> v1:172.21.15.69:6789/0 conn(0x7f02a81644e0 legacy=0x7f02a81648f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:45.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.589+0000 7f02adc8e640 1 -- 172.21.15.69:0/3300361977 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a81608b0 con 0x7f02a80b8ca0 2024-04-25T11:00:45.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.589+0000 7f02acc8c640 1 -- 172.21.15.69:0/3300361977 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 789165342 0 0) 0x7f02a806ed60 con 0x7f02a81644e0 2024-04-25T11:00:45.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.589+0000 7f02adc8e640 1 --2- 172.21.15.69:0/3300361977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02a80b8ca0 0x7f02a8163fa0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f02a40097a0 tx=0x7f02a402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a14c3af2a23bba3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:45.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.590+0000 7f02acc8c640 1 -- 172.21.15.69:0/3300361977 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a4037070 con 0x7f02a80b8ca0 2024-04-25T11:00:45.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.590+0000 7f02acc8c640 1 -- 172.21.15.69:0/3300361977 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02a4032440 con 0x7f02a80b8ca0 2024-04-25T11:00:45.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.590+0000 7f02acc8c640 1 -- 172.21.15.69:0/3300361977 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a4032810 con 0x7f02a80b8ca0 2024-04-25T11:00:45.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.590+0000 7f02afef2640 1 -- 172.21.15.69:0/3300361977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02a80b8ca0 msgr2=0x7f02a8163fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:45.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.590+0000 7f02afef2640 1 --2- 172.21.15.69:0/3300361977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02a80b8ca0 0x7f02a8163fa0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f02a40097a0 tx=0x7f02a402a180 comp rx=0 tx=0).stop 2024-04-25T11:00:45.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.590+0000 7f02afef2640 1 -- 172.21.15.69:0/3300361977 shutdown_connections 2024-04-25T11:00:45.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.590+0000 7f02afef2640 1 --2- 172.21.15.69:0/3300361977 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02a80ce060 0x7f02a80ce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:45.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.590+0000 7f02afef2640 1 --2- 172.21.15.69:0/3300361977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02a80b8ca0 0x7f02a8163fa0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:45.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.590+0000 7f02afef2640 1 -- 172.21.15.69:0/3300361977 >> 172.21.15.69:0/3300361977 conn(0x7f02a80d3aa0 msgr2=0x7f02a80d3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:45.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.590+0000 7f02afef2640 1 -- 172.21.15.69:0/3300361977 shutdown_connections 2024-04-25T11:00:45.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.590+0000 7f02afef2640 1 -- 172.21.15.69:0/3300361977 wait complete. 2024-04-25T11:00:45.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.592+0000 7f02afef2640 1 Processor -- start 2024-04-25T11:00:45.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.592+0000 7f02afef2640 1 -- start start 2024-04-25T11:00:45.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.593+0000 7f02afef2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02a80b8ca0 0x7f02a81eac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:45.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.593+0000 7f02afef2640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02a80ce060 0x7f02a815db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:45.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.593+0000 7f02afef2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02a81644e0 0x7f02a815ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:45.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.593+0000 7f02afef2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f02a806e560 con 0x7f02a81644e0 2024-04-25T11:00:45.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.593+0000 7f02afef2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f02a806e280 con 0x7f02a80b8ca0 2024-04-25T11:00:45.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.593+0000 7f02afef2640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f02a80d2f40 con 0x7f02a80ce060 2024-04-25T11:00:45.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.593+0000 7f02ae48f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02a81644e0 0x7f02a815ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:45.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.594+0000 7f02ae48f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02a81644e0 0x7f02a815ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47772/0 (socket says 172.21.15.69:47772) 2024-04-25T11:00:45.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.594+0000 7f02ae48f640 1 -- 172.21.15.69:0/4198215882 learned_addr learned my addr 172.21.15.69:0/4198215882 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:45.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.594+0000 7f02ad48d640 1 --2- 172.21.15.69:0/4198215882 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02a80ce060 0x7f02a815db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:45.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.594+0000 7f02adc8e640 1 --2- 172.21.15.69:0/4198215882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02a80b8ca0 0x7f02a81eac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:45.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.594+0000 7f02ae48f640 1 -- 172.21.15.69:0/4198215882 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02a80ce060 msgr2=0x7f02a815db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:45.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.594+0000 7f02ae48f640 1 --2- 172.21.15.69:0/4198215882 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02a80ce060 0x7f02a815db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:45.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.594+0000 7f02ae48f640 1 -- 172.21.15.69:0/4198215882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02a80b8ca0 msgr2=0x7f02a81eac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:45.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.594+0000 7f02ae48f640 1 --2- 172.21.15.69:0/4198215882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02a80b8ca0 0x7f02a81eac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:45.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.594+0000 7f02ae48f640 1 -- 172.21.15.69:0/4198215882 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a80c9810 con 0x7f02a81644e0 2024-04-25T11:00:45.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.594+0000 7f02ae48f640 1 --2- 172.21.15.69:0/4198215882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02a81644e0 0x7f02a815ef20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f02a000cc30 tx=0x7f02a000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:45.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.594+0000 7f029e7fc640 1 -- 172.21.15.69:0/4198215882 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a000ce40 con 0x7f02a81644e0 2024-04-25T11:00:45.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.594+0000 7f029e7fc640 1 -- 172.21.15.69:0/4198215882 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02a0003990 con 0x7f02a81644e0 2024-04-25T11:00:45.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.594+0000 7f02afef2640 1 -- 172.21.15.69:0/4198215882 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02a80c9af0 con 0x7f02a81644e0 2024-04-25T11:00:45.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.594+0000 7f029e7fc640 1 -- 172.21.15.69:0/4198215882 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a0003d40 con 0x7f02a81644e0 2024-04-25T11:00:45.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.594+0000 7f02afef2640 1 -- 172.21.15.69:0/4198215882 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02a81eb1c0 con 0x7f02a81644e0 2024-04-25T11:00:45.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.595+0000 7f02afef2640 1 -- 172.21.15.69:0/4198215882 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02a80c9580 con 0x7f02a81644e0 2024-04-25T11:00:45.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.596+0000 7f029e7fc640 1 -- 172.21.15.69:0/4198215882 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f02a0003ee0 con 0x7f02a81644e0 2024-04-25T11:00:45.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.596+0000 7f029e7fc640 1 --2- 172.21.15.69:0/4198215882 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f028c0744b0 0x7f028c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:45.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.596+0000 7f029e7fc640 1 -- 172.21.15.69:0/4198215882 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7f02a0095450 con 0x7f02a81644e0 2024-04-25T11:00:45.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.596+0000 7f02adc8e640 1 --2- 172.21.15.69:0/4198215882 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f028c0744b0 0x7f028c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:45.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.597+0000 7f02adc8e640 1 --2- 172.21.15.69:0/4198215882 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f028c0744b0 0x7f028c0769c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f02a4034000 tx=0x7f02a402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:45.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.601+0000 7f029e7fc640 1 -- 172.21.15.69:0/4198215882 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f02a0014030 con 0x7f02a81644e0 2024-04-25T11:00:45.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:45.844+0000 7f02afef2640 1 -- 172.21.15.69:0/4198215882 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_cap_flush.TestCapFlush.test_replay_create"]} v 0) -- 0x7f02a8050740 con 0x7f02a81644e0 2024-04-25T11:00:46.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:46.136+0000 7f029e7fc640 1 -- 172.21.15.69:0/4198215882 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_cap_flush.TestCapFlush.test_replay_create"]}]=0 v101) ==== 138+0+0 (secure 0 0 0) 0x7f02a005fe80 con 0x7f02a81644e0 2024-04-25T11:00:46.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:46.138+0000 7f02afef2640 1 -- 172.21.15.69:0/4198215882 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f028c0744b0 msgr2=0x7f028c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:46.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:46.138+0000 7f02afef2640 1 --2- 172.21.15.69:0/4198215882 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f028c0744b0 0x7f028c0769c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f02a4034000 tx=0x7f02a402e000 comp rx=0 tx=0).stop 2024-04-25T11:00:46.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:46.138+0000 7f02afef2640 1 -- 172.21.15.69:0/4198215882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02a81644e0 msgr2=0x7f02a815ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:46.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:46.138+0000 7f02afef2640 1 --2- 172.21.15.69:0/4198215882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02a81644e0 0x7f02a815ef20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f02a000cc30 tx=0x7f02a000a5c0 comp rx=0 tx=0).stop 2024-04-25T11:00:46.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:46.138+0000 7f02afef2640 1 -- 172.21.15.69:0/4198215882 shutdown_connections 2024-04-25T11:00:46.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:46.138+0000 7f02afef2640 1 --2- 172.21.15.69:0/4198215882 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f028c0744b0 0x7f028c0769c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:46.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:46.138+0000 7f02afef2640 1 --2- 172.21.15.69:0/4198215882 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02a81644e0 0x7f02a815ef20 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:46.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:46.138+0000 7f02afef2640 1 --2- 172.21.15.69:0/4198215882 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02a80ce060 0x7f02a815db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:46.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:46.138+0000 7f02afef2640 1 --2- 172.21.15.69:0/4198215882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02a80b8ca0 0x7f02a81eac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:46.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:46.138+0000 7f02afef2640 1 -- 172.21.15.69:0/4198215882 >> 172.21.15.69:0/4198215882 conn(0x7f02a80d3aa0 msgr2=0x7f02a80d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:46.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:46.138+0000 7f02afef2640 1 -- 172.21.15.69:0/4198215882 shutdown_connections 2024-04-25T11:00:46.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:46.138+0000 7f02afef2640 1 -- 172.21.15.69:0/4198215882 wait complete. 2024-04-25T11:00:46.150 INFO:tasks.cephfs_test_runner:MDS starts to handle client caps when it enters clientreplay stage. ... ok 2024-04-25T11:00:46.150 INFO:tasks.cephfs_test_runner: 2024-04-25T11:00:46.151 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T11:00:46.151 INFO:tasks.cephfs_test_runner:Ran 1 test in 62.831s 2024-04-25T11:00:46.151 INFO:tasks.cephfs_test_runner: 2024-04-25T11:00:46.151 INFO:tasks.cephfs_test_runner:OK 2024-04-25T11:00:46.151 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-25T11:00:46.161 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T11:00:46.203 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T11:00:46.203 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T11:00:46.204 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:00:46.219 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=499 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:00:46.220 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T11:00:46.220 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:00:46.274 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=499 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:00:46.274 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T11:00:46.274 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:00:46.293 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1771 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:00:46.293 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T11:00:46.293 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:00:46.350 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1771 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:00:46.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:00:46.350 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2024-04-25T11:00:46.365 INFO:teuthology.orchestra.run.smithi069.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:00:46.365 INFO:teuthology.orchestra.run.smithi069.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:00:46.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:00:46.365 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:00:46.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:00:46.429 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:00:46.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:00:46.493 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link delete ceph-brx 2024-04-25T11:00:46.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:00:46.589 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2024-04-25T11:00:46.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:00:46.603 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link delete ceph-brx 2024-04-25T11:00:46.670 INFO:teuthology.orchestra.run.smithi069.stderr:Cannot find device "ceph-brx" 2024-04-25T11:00:46.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:00:46.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:00:46.671 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-04-25T11:00:46.689 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T11:00:46.689 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T11:00:46.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:00:46.690 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T11:00:46.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:00:46.758 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T11:00:46.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:00:46.827 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link delete ceph-brx 2024-04-25T11:00:46.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:00:46.921 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-04-25T11:00:46.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:00:46.937 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link delete ceph-brx 2024-04-25T11:00:47.003 INFO:teuthology.orchestra.run.smithi146.stderr:Cannot find device "ceph-brx" 2024-04-25T11:00:47.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:00:47.005 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T11:00:47.018 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T11:00:47.019 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:00:47.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.127+0000 7f5fa1e96640 1 Processor -- start 2024-04-25T11:00:47.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.127+0000 7f5fa1e96640 1 -- start start 2024-04-25T11:00:47.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.128+0000 7f5fa1e96640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5f9c0d0e00 0x7f5f9c0d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:47.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.128+0000 7f5fa1e96640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f9c0d1850 con 0x7f5f9c06c5c0 2024-04-25T11:00:47.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.128+0000 7f5fa1e96640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f9c160de0 con 0x7f5f9c0d4a50 2024-04-25T11:00:47.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.128+0000 7f5fa1e96640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f5f9c1610c0 con 0x7f5f9c0d0e00 2024-04-25T11:00:47.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.128+0000 7f5f9affd640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f5f9c06c5c0 0x7f5f9c06c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:54394/0 (socket says 172.21.15.69:54394) 2024-04-25T11:00:47.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.128+0000 7f5f9affd640 1 -- 172.21.15.69:0/2368099422 learned_addr learned my addr 172.21.15.69:0/2368099422 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:47.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.128+0000 7f5f9a7fc640 1 -- 172.21.15.69:0/2368099422 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2739276983 0 0) 0x7f5f9c0d1850 con 0x7f5f9c06c5c0 2024-04-25T11:00:47.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.128+0000 7f5f9b7fe640 1 --2- 172.21.15.69:0/2368099422 >> v2:172.21.15.146:3300/0 conn(0x7f5f9c0d0e00 0x7f5f9c0d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:47.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.128+0000 7f5f9a7fc640 1 -- 172.21.15.69:0/2368099422 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f88003520 con 0x7f5f9c06c5c0 2024-04-25T11:00:47.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.129+0000 7f5f9a7fc640 1 -- 172.21.15.69:0/2368099422 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2816069651 0 0) 0x7f5f88003520 con 0x7f5f9c06c5c0 2024-04-25T11:00:47.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.129+0000 7f5f9a7fc640 1 -- 172.21.15.69:0/2368099422 >> v2:172.21.15.146:3300/0 conn(0x7f5f9c0d0e00 msgr2=0x7f5f9c0d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:47.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.129+0000 7f5f9a7fc640 1 --2- 172.21.15.69:0/2368099422 >> v2:172.21.15.146:3300/0 conn(0x7f5f9c0d0e00 0x7f5f9c0d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:47.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.129+0000 7f5f9a7fc640 1 -- 172.21.15.69:0/2368099422 >> v1:172.21.15.146:6789/0 conn(0x7f5f9c0d4a50 legacy=0x7f5f9c0d4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:47.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.129+0000 7f5f9a7fc640 1 -- 172.21.15.69:0/2368099422 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f9c1613a0 con 0x7f5f9c06c5c0 2024-04-25T11:00:47.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.129+0000 7f5f9a7fc640 1 -- 172.21.15.69:0/2368099422 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7f5f84001670 con 0x7f5f9c06c5c0 2024-04-25T11:00:47.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.129+0000 7f5f9a7fc640 1 -- 172.21.15.69:0/2368099422 >> v1:172.21.15.69:6789/0 conn(0x7f5f9c06c5c0 legacy=0x7f5f9c06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:47.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.129+0000 7f5f9a7fc640 1 --2- 172.21.15.69:0/2368099422 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f880039b0 0x7f5f88003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:47.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.129+0000 7f5f9a7fc640 1 --2- 172.21.15.69:0/2368099422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f88004800 0x7f5f88004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:47.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.129+0000 7f5f9a7fc640 1 --2- 172.21.15.69:0/2368099422 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f880051c0 0x7f5f8800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:47.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.129+0000 7f5f9a7fc640 1 -- 172.21.15.69:0/2368099422 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9c1610c0 con 0x7f5f88004800 2024-04-25T11:00:47.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.129+0000 7f5f9a7fc640 1 -- 172.21.15.69:0/2368099422 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8800c160 con 0x7f5f880039b0 2024-04-25T11:00:47.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.129+0000 7f5f9a7fc640 1 -- 172.21.15.69:0/2368099422 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5f8800c360 con 0x7f5f880051c0 2024-04-25T11:00:47.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.129+0000 7f5f9affd640 1 --2- 172.21.15.69:0/2368099422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f88004800 0x7f5f88004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:47.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.129+0000 7f5f9b7fe640 1 --2- 172.21.15.69:0/2368099422 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f880039b0 0x7f5f88003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:47.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.129+0000 7f5f9bfff640 1 --2- 172.21.15.69:0/2368099422 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f880051c0 0x7f5f8800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:47.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.129+0000 7f5f9affd640 1 -- 172.21.15.69:0/2368099422 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f880051c0 msgr2=0x7f5f8800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:47.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.129+0000 7f5f9affd640 1 --2- 172.21.15.69:0/2368099422 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f880051c0 0x7f5f8800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:47.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.129+0000 7f5f9affd640 1 -- 172.21.15.69:0/2368099422 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f880039b0 msgr2=0x7f5f88003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:47.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.129+0000 7f5f9affd640 1 --2- 172.21.15.69:0/2368099422 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f880039b0 0x7f5f88003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:47.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.129+0000 7f5f9affd640 1 -- 172.21.15.69:0/2368099422 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f88003520 con 0x7f5f88004800 2024-04-25T11:00:47.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.129+0000 7f5f9bfff640 1 -- 172.21.15.69:0/2368099422 reap_dead start 2024-04-25T11:00:47.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.130+0000 7f5f9affd640 1 --2- 172.21.15.69:0/2368099422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f88004800 0x7f5f88004c10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400c240 tx=0x7f5f88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2773a8a616bc5dc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:47.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.130+0000 7f5f9a7fc640 1 -- 172.21.15.69:0/2368099422 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f84007d10 con 0x7f5f88004800 2024-04-25T11:00:47.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.130+0000 7f5f9a7fc640 1 -- 172.21.15.69:0/2368099422 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f84005ad0 con 0x7f5f88004800 2024-04-25T11:00:47.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.130+0000 7f5f9a7fc640 1 -- 172.21.15.69:0/2368099422 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f84007070 con 0x7f5f88004800 2024-04-25T11:00:47.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.131+0000 7f5fa1e96640 1 -- 172.21.15.69:0/2368099422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f88004800 msgr2=0x7f5f88004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:47.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.131+0000 7f5fa1e96640 1 --2- 172.21.15.69:0/2368099422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f88004800 0x7f5f88004c10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400c240 tx=0x7f5f88005830 comp rx=0 tx=0).stop 2024-04-25T11:00:47.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.131+0000 7f5fa1e96640 1 -- 172.21.15.69:0/2368099422 shutdown_connections 2024-04-25T11:00:47.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.131+0000 7f5fa1e96640 1 --2- 172.21.15.69:0/2368099422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f88004800 0x7f5f88004c10 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:47.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.131+0000 7f5fa1e96640 1 -- 172.21.15.69:0/2368099422 >> 172.21.15.69:0/2368099422 conn(0x7f5f9c0d5d90 msgr2=0x7f5f9c0d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:47.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.131+0000 7f5fa1e96640 1 -- 172.21.15.69:0/2368099422 shutdown_connections 2024-04-25T11:00:47.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.131+0000 7f5fa1e96640 1 -- 172.21.15.69:0/2368099422 wait complete. 2024-04-25T11:00:47.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.133+0000 7f5fa1e96640 1 Processor -- start 2024-04-25T11:00:47.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.133+0000 7f5fa1e96640 1 -- start start 2024-04-25T11:00:47.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.134+0000 7f5fa1e96640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f88004800 0x7f5f9c155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:47.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.134+0000 7f5fa1e96640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f9c0cab50 0x7f5f9c163570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:47.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.134+0000 7f5fa1e96640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f9c163ab0 0x7f5f9c1619a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:47.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.134+0000 7f5fa1e96640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9c164e00 con 0x7f5f88004800 2024-04-25T11:00:47.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.134+0000 7f5fa1e96640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9c0b5320 con 0x7f5f9c163ab0 2024-04-25T11:00:47.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.134+0000 7f5fa1e96640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5f9c164f80 con 0x7f5f9c0cab50 2024-04-25T11:00:47.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.134+0000 7f5f9b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f88004800 0x7f5f9c155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:47.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.134+0000 7f5f9b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f88004800 0x7f5f9c155320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47784/0 (socket says 172.21.15.69:47784) 2024-04-25T11:00:47.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.134+0000 7f5f9b7fe640 1 -- 172.21.15.69:0/513186998 learned_addr learned my addr 172.21.15.69:0/513186998 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:47.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.134+0000 7f5f9bfff640 1 --2- 172.21.15.69:0/513186998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f9c163ab0 0x7f5f9c1619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:47.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.134+0000 7f5f9affd640 1 --2- 172.21.15.69:0/513186998 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f9c0cab50 0x7f5f9c163570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:47.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.134+0000 7f5f9b7fe640 1 -- 172.21.15.69:0/513186998 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f9c0cab50 msgr2=0x7f5f9c163570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:47.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.134+0000 7f5f9b7fe640 1 --2- 172.21.15.69:0/513186998 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f9c0cab50 0x7f5f9c163570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:47.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.134+0000 7f5f9b7fe640 1 -- 172.21.15.69:0/513186998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f9c163ab0 msgr2=0x7f5f9c1619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:47.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.134+0000 7f5f9b7fe640 1 --2- 172.21.15.69:0/513186998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f9c163ab0 0x7f5f9c1619a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:47.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.134+0000 7f5f9b7fe640 1 -- 172.21.15.69:0/513186998 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f9c163ec0 con 0x7f5f88004800 2024-04-25T11:00:47.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.135+0000 7f5f9b7fe640 1 --2- 172.21.15.69:0/513186998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f88004800 0x7f5f9c155320 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c009fc0 tx=0x7f5f8c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:47.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.135+0000 7f5f93fff640 1 -- 172.21.15.69:0/513186998 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c015430 con 0x7f5f88004800 2024-04-25T11:00:47.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.135+0000 7f5f93fff640 1 -- 172.21.15.69:0/513186998 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f8c0155d0 con 0x7f5f88004800 2024-04-25T11:00:47.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.135+0000 7f5fa1e96640 1 -- 172.21.15.69:0/513186998 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f9c0cb5d0 con 0x7f5f88004800 2024-04-25T11:00:47.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.135+0000 7f5f93fff640 1 -- 172.21.15.69:0/513186998 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c0159c0 con 0x7f5f88004800 2024-04-25T11:00:47.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.135+0000 7f5fa1e96640 1 -- 172.21.15.69:0/513186998 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f9c0cbb90 con 0x7f5f88004800 2024-04-25T11:00:47.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.136+0000 7f5fa1e96640 1 -- 172.21.15.69:0/513186998 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f9c0d4830 con 0x7f5f88004800 2024-04-25T11:00:47.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.137+0000 7f5f93fff640 1 -- 172.21.15.69:0/513186998 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5f8c015b60 con 0x7f5f88004800 2024-04-25T11:00:47.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.137+0000 7f5f93fff640 1 --2- 172.21.15.69:0/513186998 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5f7c0744e0 0x7f5f7c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:47.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.137+0000 7f5f9affd640 1 --2- 172.21.15.69:0/513186998 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5f7c0744e0 0x7f5f7c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:47.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.137+0000 7f5f93fff640 1 -- 172.21.15.69:0/513186998 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7f5f8c096060 con 0x7f5f88004800 2024-04-25T11:00:47.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.138+0000 7f5f9affd640 1 --2- 172.21.15.69:0/513186998 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5f7c0744e0 0x7f5f7c0769f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5f84032000 tx=0x7f5f84009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:47.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.144+0000 7f5f93fff640 1 -- 172.21.15.69:0/513186998 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5f9c163ec0 con 0x7f5f88004800 2024-04-25T11:00:47.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.317+0000 7f5fa1e96640 1 -- 172.21.15.69:0/513186998 --> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5f9c0d63b0 con 0x7f5f7c0744e0 2024-04-25T11:00:47.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.318+0000 7f5f93fff640 1 -- 172.21.15.69:0/513186998 <== mgr.4107 v2:172.21.15.69:6832/1107421267 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13365 (secure 0 0 0) 0x7f5f9c0d63b0 con 0x7f5f7c0744e0 2024-04-25T11:00:47.319 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:00:47.319 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-04-25T11:00:47.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.320+0000 7f5fa1e96640 1 -- 172.21.15.69:0/513186998 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5f7c0744e0 msgr2=0x7f5f7c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:47.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.320+0000 7f5fa1e96640 1 --2- 172.21.15.69:0/513186998 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5f7c0744e0 0x7f5f7c0769f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5f84032000 tx=0x7f5f84009030 comp rx=0 tx=0).stop 2024-04-25T11:00:47.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.320+0000 7f5fa1e96640 1 -- 172.21.15.69:0/513186998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f88004800 msgr2=0x7f5f9c155320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:47.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.320+0000 7f5fa1e96640 1 --2- 172.21.15.69:0/513186998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f88004800 0x7f5f9c155320 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c009fc0 tx=0x7f5f8c00f580 comp rx=0 tx=0).stop 2024-04-25T11:00:47.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.320+0000 7f5fa1e96640 1 -- 172.21.15.69:0/513186998 shutdown_connections 2024-04-25T11:00:47.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.320+0000 7f5fa1e96640 1 --2- 172.21.15.69:0/513186998 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5f7c0744e0 0x7f5f7c0769f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:47.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.320+0000 7f5fa1e96640 1 --2- 172.21.15.69:0/513186998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f9c163ab0 0x7f5f9c1619a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:47.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.320+0000 7f5fa1e96640 1 --2- 172.21.15.69:0/513186998 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f9c0cab50 0x7f5f9c163570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:47.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.320+0000 7f5fa1e96640 1 --2- 172.21.15.69:0/513186998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f88004800 0x7f5f9c155320 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:47.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.320+0000 7f5fa1e96640 1 -- 172.21.15.69:0/513186998 >> 172.21.15.69:0/513186998 conn(0x7f5f9c0d5d90 msgr2=0x7f5f9c0bb480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:47.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.320+0000 7f5fa1e96640 1 -- 172.21.15.69:0/513186998 shutdown_connections 2024-04-25T11:00:47.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.320+0000 7f5fa1e96640 1 -- 172.21.15.69:0/513186998 wait complete. 2024-04-25T11:00:47.331 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2024-04-25T11:00:46.423151+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220220,"kb_used_data":4072,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754500,"statfs":{"total":773094113280,"available":772868608000,"internally_reserved":0,"allocated":4169728,"data_stored":2341406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16341,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-86,"num_read_kb":-88,"num_write":-74,"num_write_kb":-42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000883"},"pg_stats":[{"pgid":"1.0","version":"15'72","reported_seq":131,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:00:30.092907+0000","last_change":"2024-04-25T10:59:07.839466+0000","last_active":"2024-04-25T11:00:30.092907+0000","last_peered":"2024-04-25T11:00:30.092907+0000","last_clean":"2024-04-25T11:00:30.092907+0000","last_became_active":"2024-04-25T10:59:07.836269+0000","last_became_peered":"2024-04-25T10:59:07.836269+0000","last_unstale":"2024-04-25T11:00:30.092907+0000","last_undegraded":"2024-04-25T11:00:30.092907+0000","last_fullsized":"2024-04-25T11:00:30.092907+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:06.549663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:06.549663+0000","last_clean_scrub_stamp":"2024-04-25T10:59:06.549663+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:21.575330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":958464,"data_stored":730989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":141147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":141203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":140635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":393216,"data_stored":157966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":364544,"data_stored":140601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":393216,"data_stored":157966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":954368,"data_stored":730899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:00:47.332 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:00:47.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.438+0000 7f5224140640 1 Processor -- start 2024-04-25T11:00:47.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.439+0000 7f5224140640 1 -- start start 2024-04-25T11:00:47.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.440+0000 7f5224140640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f521c0b21a0 0x7f521c0ce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:47.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.440+0000 7f5224140640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f521c0d1c10 con 0x7f521c0b7840 2024-04-25T11:00:47.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.440+0000 7f5224140640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f521c0b25b0 con 0x7f521c0cfd40 2024-04-25T11:00:47.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.440+0000 7f5224140640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f521c0d27e0 con 0x7f521c0b21a0 2024-04-25T11:00:47.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.440+0000 7f52226dd640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f521c0b7840 0x7f521c0b7c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:54406/0 (socket says 172.21.15.69:54406) 2024-04-25T11:00:47.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.440+0000 7f52226dd640 1 -- 172.21.15.69:0/1128785119 learned_addr learned my addr 172.21.15.69:0/1128785119 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:47.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.440+0000 7f52216db640 1 --2- 172.21.15.69:0/1128785119 >> v2:172.21.15.146:3300/0 conn(0x7f521c0b21a0 0x7f521c0ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:47.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.440+0000 7f5220eda640 1 -- 172.21.15.69:0/1128785119 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2247122462 0 0) 0x7f521c0d1c10 con 0x7f521c0b7840 2024-04-25T11:00:47.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.440+0000 7f5220eda640 1 -- 172.21.15.69:0/1128785119 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5208003520 con 0x7f521c0b7840 2024-04-25T11:00:47.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.441+0000 7f5220eda640 1 -- 172.21.15.69:0/1128785119 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2787435967 0 0) 0x7f521c0b25b0 con 0x7f521c0cfd40 2024-04-25T11:00:47.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.441+0000 7f5220eda640 1 -- 172.21.15.69:0/1128785119 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f521c0d1c10 con 0x7f521c0cfd40 2024-04-25T11:00:47.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.441+0000 7f5220eda640 1 -- 172.21.15.69:0/1128785119 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2928950686 0 0) 0x7f5208003520 con 0x7f521c0b7840 2024-04-25T11:00:47.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.441+0000 7f5220eda640 1 -- 172.21.15.69:0/1128785119 >> v2:172.21.15.146:3300/0 conn(0x7f521c0b21a0 msgr2=0x7f521c0ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:47.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.441+0000 7f5220eda640 1 --2- 172.21.15.69:0/1128785119 >> v2:172.21.15.146:3300/0 conn(0x7f521c0b21a0 0x7f521c0ce110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:47.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.441+0000 7f5220eda640 1 -- 172.21.15.69:0/1128785119 >> v1:172.21.15.146:6789/0 conn(0x7f521c0cfd40 legacy=0x7f521c0b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:47.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.441+0000 7f5220eda640 1 -- 172.21.15.69:0/1128785119 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f521c0d1eb0 con 0x7f521c0b7840 2024-04-25T11:00:47.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.441+0000 7f5220eda640 1 -- 172.21.15.69:0/1128785119 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7f5214001690 con 0x7f521c0b7840 2024-04-25T11:00:47.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.441+0000 7f5220eda640 1 -- 172.21.15.69:0/1128785119 >> v1:172.21.15.69:6789/0 conn(0x7f521c0b7840 legacy=0x7f521c0b7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:47.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.441+0000 7f5220eda640 1 --2- 172.21.15.69:0/1128785119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5208003b40 0x7f5208003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:47.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.441+0000 7f5220eda640 1 --2- 172.21.15.69:0/1128785119 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5208004800 0x7f5208004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:47.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.441+0000 7f5220eda640 1 --2- 172.21.15.69:0/1128785119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5208005260 0x7f520800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:47.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.441+0000 7f5220eda640 1 -- 172.21.15.69:0/1128785119 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f521c0d27e0 con 0x7f5208003b40 2024-04-25T11:00:47.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.441+0000 7f5220eda640 1 -- 172.21.15.69:0/1128785119 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f520800c1c0 con 0x7f5208005260 2024-04-25T11:00:47.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.441+0000 7f5220eda640 1 -- 172.21.15.69:0/1128785119 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f520800c3c0 con 0x7f5208004800 2024-04-25T11:00:47.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.441+0000 7f5221edc640 1 --2- 172.21.15.69:0/1128785119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5208003b40 0x7f5208003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:47.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.441+0000 7f52216db640 1 --2- 172.21.15.69:0/1128785119 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5208004800 0x7f5208004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:47.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.441+0000 7f52226dd640 1 --2- 172.21.15.69:0/1128785119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5208005260 0x7f520800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:47.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.441+0000 7f5221edc640 1 -- 172.21.15.69:0/1128785119 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5208004800 msgr2=0x7f5208004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:47.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.441+0000 7f5221edc640 1 --2- 172.21.15.69:0/1128785119 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5208004800 0x7f5208004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:47.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.441+0000 7f5221edc640 1 -- 172.21.15.69:0/1128785119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5208005260 msgr2=0x7f520800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:47.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.441+0000 7f5221edc640 1 --2- 172.21.15.69:0/1128785119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5208005260 0x7f520800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:47.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.441+0000 7f5221edc640 1 -- 172.21.15.69:0/1128785119 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5208003520 con 0x7f5208003b40 2024-04-25T11:00:47.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.441+0000 7f52226dd640 1 --2- 172.21.15.69:0/1128785119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5208005260 0x7f520800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:47.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.441+0000 7f52226dd640 1 -- 172.21.15.69:0/1128785119 reap_dead start 2024-04-25T11:00:47.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.442+0000 7f5221edc640 1 --2- 172.21.15.69:0/1128785119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5208003b40 0x7f5208003f90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5218001fd0 tx=0x7f5208005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c973f795133a2f33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:47.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.442+0000 7f5220eda640 1 -- 172.21.15.69:0/1128785119 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5218001bf0 con 0x7f5208003b40 2024-04-25T11:00:47.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.442+0000 7f5220eda640 1 -- 172.21.15.69:0/1128785119 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5218003b70 con 0x7f5208003b40 2024-04-25T11:00:47.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.442+0000 7f5220eda640 1 -- 172.21.15.69:0/1128785119 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52180050d0 con 0x7f5208003b40 2024-04-25T11:00:47.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.443+0000 7f5224140640 1 -- 172.21.15.69:0/1128785119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5208003b40 msgr2=0x7f5208003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:47.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.443+0000 7f5224140640 1 --2- 172.21.15.69:0/1128785119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5208003b40 0x7f5208003f90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5218001fd0 tx=0x7f5208005830 comp rx=0 tx=0).stop 2024-04-25T11:00:47.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.443+0000 7f5224140640 1 -- 172.21.15.69:0/1128785119 shutdown_connections 2024-04-25T11:00:47.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.443+0000 7f5224140640 1 --2- 172.21.15.69:0/1128785119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5208003b40 0x7f5208003f90 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:47.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.443+0000 7f5224140640 1 -- 172.21.15.69:0/1128785119 >> 172.21.15.69:0/1128785119 conn(0x7f521c09b7d0 msgr2=0x7f521c09bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:47.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.443+0000 7f5224140640 1 -- 172.21.15.69:0/1128785119 shutdown_connections 2024-04-25T11:00:47.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.443+0000 7f5224140640 1 -- 172.21.15.69:0/1128785119 wait complete. 2024-04-25T11:00:47.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.444+0000 7f5224140640 1 Processor -- start 2024-04-25T11:00:47.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.445+0000 7f5224140640 1 -- start start 2024-04-25T11:00:47.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.446+0000 7f5224140640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5208003b40 0x7f521c161bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:47.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.446+0000 7f5224140640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f521c162100 0x7f521c15ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:47.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.446+0000 7f5224140640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f521c160470 0x7f521c155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:47.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.446+0000 7f5224140640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f521c0b3e00 con 0x7f5208003b40 2024-04-25T11:00:47.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.446+0000 7f5224140640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f521c0d2470 con 0x7f521c160470 2024-04-25T11:00:47.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.446+0000 7f5224140640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f521c0b3f80 con 0x7f521c162100 2024-04-25T11:00:47.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.446+0000 7f5221edc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5208003b40 0x7f521c161bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:47.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.446+0000 7f5221edc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5208003b40 0x7f521c161bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47804/0 (socket says 172.21.15.69:47804) 2024-04-25T11:00:47.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.446+0000 7f5221edc640 1 -- 172.21.15.69:0/1568749752 learned_addr learned my addr 172.21.15.69:0/1568749752 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:47.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.446+0000 7f5221edc640 1 -- 172.21.15.69:0/1568749752 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f521c162100 msgr2=0x7f521c15ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:47.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.446+0000 7f52216db640 1 --2- 172.21.15.69:0/1568749752 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f521c162100 0x7f521c15ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:47.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.446+0000 7f5221edc640 1 --2- 172.21.15.69:0/1568749752 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f521c162100 0x7f521c15ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:47.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.446+0000 7f52226dd640 1 --2- 172.21.15.69:0/1568749752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f521c160470 0x7f521c155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:47.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.446+0000 7f5221edc640 1 -- 172.21.15.69:0/1568749752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f521c160470 msgr2=0x7f521c155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:47.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.446+0000 7f5221edc640 1 --2- 172.21.15.69:0/1568749752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f521c160470 0x7f521c155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:47.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.446+0000 7f5221edc640 1 -- 172.21.15.69:0/1568749752 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f521c0cb480 con 0x7f5208003b40 2024-04-25T11:00:47.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.446+0000 7f5221edc640 1 --2- 172.21.15.69:0/1568749752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5208003b40 0x7f521c161bc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5218033040 tx=0x7f5218001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:47.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.446+0000 7f52127fc640 1 -- 172.21.15.69:0/1568749752 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5218040070 con 0x7f5208003b40 2024-04-25T11:00:47.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.446+0000 7f52127fc640 1 -- 172.21.15.69:0/1568749752 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5218003970 con 0x7f5208003b40 2024-04-25T11:00:47.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.446+0000 7f5224140640 1 -- 172.21.15.69:0/1568749752 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f521c160c20 con 0x7f5208003b40 2024-04-25T11:00:47.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.447+0000 7f52127fc640 1 -- 172.21.15.69:0/1568749752 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5218001bf0 con 0x7f5208003b40 2024-04-25T11:00:47.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.447+0000 7f5224140640 1 -- 172.21.15.69:0/1568749752 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f521c1611e0 con 0x7f5208003b40 2024-04-25T11:00:47.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.447+0000 7f5224140640 1 -- 172.21.15.69:0/1568749752 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f521c050740 con 0x7f5208003b40 2024-04-25T11:00:47.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.448+0000 7f52127fc640 1 -- 172.21.15.69:0/1568749752 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f521803a040 con 0x7f5208003b40 2024-04-25T11:00:47.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.448+0000 7f52127fc640 1 --2- 172.21.15.69:0/1568749752 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f52000744b0 0x7f52000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:47.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.449+0000 7f52216db640 1 --2- 172.21.15.69:0/1568749752 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f52000744b0 0x7f52000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:47.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.449+0000 7f52127fc640 1 -- 172.21.15.69:0/1568749752 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7f52180b5c80 con 0x7f5208003b40 2024-04-25T11:00:47.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.449+0000 7f52216db640 1 --2- 172.21.15.69:0/1568749752 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f52000744b0 0x7f52000769c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f521c0b1a30 tx=0x7f520c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:47.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.453+0000 7f52127fc640 1 -- 172.21.15.69:0/1568749752 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f52180806b0 con 0x7f5208003b40 2024-04-25T11:00:47.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.624+0000 7f5224140640 1 -- 172.21.15.69:0/1568749752 --> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f521c0cf880 con 0x7f52000744b0 2024-04-25T11:00:47.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.625+0000 7f52127fc640 1 -- 172.21.15.69:0/1568749752 <== mgr.4107 v2:172.21.15.69:6832/1107421267 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13365 (secure 0 0 0) 0x7f521c0cf880 con 0x7f52000744b0 2024-04-25T11:00:47.626 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:00:47.626 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-04-25T11:00:47.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.627+0000 7f5224140640 1 -- 172.21.15.69:0/1568749752 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f52000744b0 msgr2=0x7f52000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:47.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.627+0000 7f5224140640 1 --2- 172.21.15.69:0/1568749752 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f52000744b0 0x7f52000769c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f521c0b1a30 tx=0x7f520c008040 comp rx=0 tx=0).stop 2024-04-25T11:00:47.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.627+0000 7f5224140640 1 -- 172.21.15.69:0/1568749752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5208003b40 msgr2=0x7f521c161bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:47.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.627+0000 7f5224140640 1 --2- 172.21.15.69:0/1568749752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5208003b40 0x7f521c161bc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5218033040 tx=0x7f5218001650 comp rx=0 tx=0).stop 2024-04-25T11:00:47.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.627+0000 7f5224140640 1 -- 172.21.15.69:0/1568749752 shutdown_connections 2024-04-25T11:00:47.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.627+0000 7f5224140640 1 --2- 172.21.15.69:0/1568749752 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f52000744b0 0x7f52000769c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:47.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.627+0000 7f5224140640 1 --2- 172.21.15.69:0/1568749752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f521c160470 0x7f521c155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:47.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.627+0000 7f5224140640 1 --2- 172.21.15.69:0/1568749752 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f521c162100 0x7f521c15ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:47.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.627+0000 7f5224140640 1 --2- 172.21.15.69:0/1568749752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5208003b40 0x7f521c161bc0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:47.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.627+0000 7f5224140640 1 -- 172.21.15.69:0/1568749752 >> 172.21.15.69:0/1568749752 conn(0x7f521c09b7d0 msgr2=0x7f521c06e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:47.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.627+0000 7f5224140640 1 -- 172.21.15.69:0/1568749752 shutdown_connections 2024-04-25T11:00:47.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.627+0000 7f5224140640 1 -- 172.21.15.69:0/1568749752 wait complete. 2024-04-25T11:00:47.639 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2024-04-25T11:00:46.423151+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220220,"kb_used_data":4072,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754500,"statfs":{"total":773094113280,"available":772868608000,"internally_reserved":0,"allocated":4169728,"data_stored":2341406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16341,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-86,"num_read_kb":-88,"num_write":-74,"num_write_kb":-42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000883"},"pg_stats":[{"pgid":"1.0","version":"15'72","reported_seq":131,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:00:30.092907+0000","last_change":"2024-04-25T10:59:07.839466+0000","last_active":"2024-04-25T11:00:30.092907+0000","last_peered":"2024-04-25T11:00:30.092907+0000","last_clean":"2024-04-25T11:00:30.092907+0000","last_became_active":"2024-04-25T10:59:07.836269+0000","last_became_peered":"2024-04-25T10:59:07.836269+0000","last_unstale":"2024-04-25T11:00:30.092907+0000","last_undegraded":"2024-04-25T11:00:30.092907+0000","last_fullsized":"2024-04-25T11:00:30.092907+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:06.549663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:06.549663+0000","last_clean_scrub_stamp":"2024-04-25T10:59:06.549663+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:21.575330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":958464,"data_stored":730989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":141147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":141203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":140635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":393216,"data_stored":157966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":364544,"data_stored":140601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":393216,"data_stored":157966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":954368,"data_stored":730899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:00:47.639 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T11:00:47.639 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:00:47.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.745+0000 7f1181092640 1 Processor -- start 2024-04-25T11:00:47.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.745+0000 7f1181092640 1 -- start start 2024-04-25T11:00:47.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.746+0000 7f1181092640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f117c0cf5e0 0x7f117c0cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:47.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.746+0000 7f1181092640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f117c06c5c0 0x7f117c06c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:47.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.747+0000 7f1181092640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f117c0cff30 con 0x7f117c1443c0 2024-04-25T11:00:47.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.747+0000 7f1181092640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f117c0708d0 con 0x7f117c06c5c0 2024-04-25T11:00:47.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.747+0000 7f1181092640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f117c15f160 con 0x7f117c0cf5e0 2024-04-25T11:00:47.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.747+0000 7f117b59e640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f117c06c5c0 0x7f117c06c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:47.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.747+0000 7f117b59e640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f117c06c5c0 0x7f117c06c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47806/0 (socket says 172.21.15.69:47806) 2024-04-25T11:00:47.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.747+0000 7f117b59e640 1 -- 172.21.15.69:0/2392026726 learned_addr learned my addr 172.21.15.69:0/2392026726 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:47.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.747+0000 7f117a59c640 1 --2- 172.21.15.69:0/2392026726 >> v2:172.21.15.146:3300/0 conn(0x7f117c0cf5e0 0x7f117c0cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:47.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.747+0000 7f117b59e640 1 -- 172.21.15.69:0/2392026726 >> v2:172.21.15.146:3300/0 conn(0x7f117c0cf5e0 msgr2=0x7f117c0cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:47.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.747+0000 7f117b59e640 1 --2- 172.21.15.69:0/2392026726 >> v2:172.21.15.146:3300/0 conn(0x7f117c0cf5e0 0x7f117c0cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:47.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.747+0000 7f117b59e640 1 -- 172.21.15.69:0/2392026726 >> v1:172.21.15.146:6789/0 conn(0x7f117c1443c0 legacy=0x7f117c06fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:47.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.747+0000 7f117b59e640 1 -- 172.21.15.69:0/2392026726 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f117c15f5f0 con 0x7f117c06c5c0 2024-04-25T11:00:47.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.747+0000 7f117a59c640 1 --2- 172.21.15.69:0/2392026726 >> v2:172.21.15.146:3300/0 conn(0x7f117c0cf5e0 0x7f117c0cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:00:47.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.748+0000 7f117b59e640 1 --2- 172.21.15.69:0/2392026726 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f117c06c5c0 0x7f117c06c9d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f117c06f8f0 tx=0x7f117402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33934b9c4845a8f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:47.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.748+0000 7f1179d9b640 1 -- 172.21.15.69:0/2392026726 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1174037070 con 0x7f117c06c5c0 2024-04-25T11:00:47.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.748+0000 7f1179d9b640 1 -- 172.21.15.69:0/2392026726 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1174032440 con 0x7f117c06c5c0 2024-04-25T11:00:47.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.748+0000 7f1179d9b640 1 -- 172.21.15.69:0/2392026726 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1174032810 con 0x7f117c06c5c0 2024-04-25T11:00:47.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.748+0000 7f1181092640 1 -- 172.21.15.69:0/2392026726 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f117c06c5c0 msgr2=0x7f117c06c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:47.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.749+0000 7f1181092640 1 --2- 172.21.15.69:0/2392026726 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f117c06c5c0 0x7f117c06c9d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f117c06f8f0 tx=0x7f117402a270 comp rx=0 tx=0).stop 2024-04-25T11:00:47.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.749+0000 7f1181092640 1 -- 172.21.15.69:0/2392026726 shutdown_connections 2024-04-25T11:00:47.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.749+0000 7f1181092640 1 --2- 172.21.15.69:0/2392026726 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f117c06c5c0 0x7f117c06c9d0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:47.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.749+0000 7f1181092640 1 --2- 172.21.15.69:0/2392026726 >> v2:172.21.15.146:3300/0 conn(0x7f117c0cf5e0 0x7f117c0cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:47.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.749+0000 7f1181092640 1 -- 172.21.15.69:0/2392026726 >> 172.21.15.69:0/2392026726 conn(0x7f117c0d1ae0 msgr2=0x7f117c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:47.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.749+0000 7f1181092640 1 -- 172.21.15.69:0/2392026726 shutdown_connections 2024-04-25T11:00:47.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.749+0000 7f1181092640 1 -- 172.21.15.69:0/2392026726 wait complete. 2024-04-25T11:00:47.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.751+0000 7f1181092640 1 Processor -- start 2024-04-25T11:00:47.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.751+0000 7f1181092640 1 -- start start 2024-04-25T11:00:47.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.752+0000 7f1181092640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f117c06c5c0 0x7f117c1581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:47.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.752+0000 7f1181092640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f117c0cf5e0 0x7f117c0aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:47.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.752+0000 7f1181092640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f117c1443c0 0x7f117c159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:47.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.752+0000 7f1181092640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f117c15edf0 con 0x7f117c1443c0 2024-04-25T11:00:47.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.752+0000 7f1181092640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f117c15eb10 con 0x7f117c06c5c0 2024-04-25T11:00:47.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.752+0000 7f1181092640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f117c1447d0 con 0x7f117c0cf5e0 2024-04-25T11:00:47.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.752+0000 7f117b59e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f117c1443c0 0x7f117c159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:47.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.752+0000 7f117b59e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f117c1443c0 0x7f117c159ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47810/0 (socket says 172.21.15.69:47810) 2024-04-25T11:00:47.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.752+0000 7f117b59e640 1 -- 172.21.15.69:0/2047664071 learned_addr learned my addr 172.21.15.69:0/2047664071 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:47.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.752+0000 7f117a59c640 1 --2- 172.21.15.69:0/2047664071 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f117c0cf5e0 0x7f117c0aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:47.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.752+0000 7f117ad9d640 1 --2- 172.21.15.69:0/2047664071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f117c06c5c0 0x7f117c1581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:47.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.752+0000 7f117b59e640 1 -- 172.21.15.69:0/2047664071 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f117c0cf5e0 msgr2=0x7f117c0aa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:47.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.752+0000 7f117b59e640 1 --2- 172.21.15.69:0/2047664071 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f117c0cf5e0 0x7f117c0aa6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:47.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.752+0000 7f117b59e640 1 -- 172.21.15.69:0/2047664071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f117c06c5c0 msgr2=0x7f117c1581a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:47.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.752+0000 7f117b59e640 1 --2- 172.21.15.69:0/2047664071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f117c06c5c0 0x7f117c1581a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:47.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.752+0000 7f117b59e640 1 -- 172.21.15.69:0/2047664071 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f117c158a10 con 0x7f117c1443c0 2024-04-25T11:00:47.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.753+0000 7f117b59e640 1 --2- 172.21.15.69:0/2047664071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f117c1443c0 0x7f117c159ab0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f117402a2b0 tx=0x7f11740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:47.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.753+0000 7f11637fe640 1 -- 172.21.15.69:0/2047664071 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1174041020 con 0x7f117c1443c0 2024-04-25T11:00:47.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.753+0000 7f11637fe640 1 -- 172.21.15.69:0/2047664071 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f117403d640 con 0x7f117c1443c0 2024-04-25T11:00:47.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.753+0000 7f11637fe640 1 -- 172.21.15.69:0/2047664071 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1174037040 con 0x7f117c1443c0 2024-04-25T11:00:47.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.753+0000 7f1181092640 1 -- 172.21.15.69:0/2047664071 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f117c158cf0 con 0x7f117c1443c0 2024-04-25T11:00:47.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.753+0000 7f1181092640 1 -- 172.21.15.69:0/2047664071 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f117c1592b0 con 0x7f117c1443c0 2024-04-25T11:00:47.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.754+0000 7f1181092640 1 -- 172.21.15.69:0/2047664071 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f117c070580 con 0x7f117c1443c0 2024-04-25T11:00:47.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.755+0000 7f11637fe640 1 -- 172.21.15.69:0/2047664071 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1174044020 con 0x7f117c1443c0 2024-04-25T11:00:47.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.755+0000 7f11637fe640 1 --2- 172.21.15.69:0/2047664071 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f11680744b0 0x7f11680769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:47.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.755+0000 7f117ad9d640 1 --2- 172.21.15.69:0/2047664071 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f11680744b0 0x7f11680769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:47.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.755+0000 7f11637fe640 1 -- 172.21.15.69:0/2047664071 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7f11740b56f0 con 0x7f117c1443c0 2024-04-25T11:00:47.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.755+0000 7f117ad9d640 1 --2- 172.21.15.69:0/2047664071 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f11680744b0 0x7f11680769c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f117c0d1860 tx=0x7f116c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:47.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.760+0000 7f11637fe640 1 -- 172.21.15.69:0/2047664071 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f117c070580 con 0x7f117c1443c0 2024-04-25T11:00:47.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.930+0000 7f1181092640 1 -- 172.21.15.69:0/2047664071 --> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f117c1587a0 con 0x7f11680744b0 2024-04-25T11:00:47.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.931+0000 7f11637fe640 1 -- 172.21.15.69:0/2047664071 <== mgr.4107 v2:172.21.15.69:6832/1107421267 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13365 (secure 0 0 0) 0x7f117c1587a0 con 0x7f11680744b0 2024-04-25T11:00:47.932 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:00:47.932 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-04-25T11:00:47.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.933+0000 7f1181092640 1 -- 172.21.15.69:0/2047664071 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f11680744b0 msgr2=0x7f11680769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:47.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.933+0000 7f1181092640 1 --2- 172.21.15.69:0/2047664071 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f11680744b0 0x7f11680769c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f117c0d1860 tx=0x7f116c009040 comp rx=0 tx=0).stop 2024-04-25T11:00:47.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.933+0000 7f1181092640 1 -- 172.21.15.69:0/2047664071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f117c1443c0 msgr2=0x7f117c159ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:47.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.933+0000 7f1181092640 1 --2- 172.21.15.69:0/2047664071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f117c1443c0 0x7f117c159ab0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f117402a2b0 tx=0x7f11740046c0 comp rx=0 tx=0).stop 2024-04-25T11:00:47.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.933+0000 7f1181092640 1 -- 172.21.15.69:0/2047664071 shutdown_connections 2024-04-25T11:00:47.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.933+0000 7f1181092640 1 --2- 172.21.15.69:0/2047664071 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f11680744b0 0x7f11680769c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:47.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.933+0000 7f1181092640 1 --2- 172.21.15.69:0/2047664071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f117c1443c0 0x7f117c159ab0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:47.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.933+0000 7f1181092640 1 --2- 172.21.15.69:0/2047664071 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f117c0cf5e0 0x7f117c0aa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:47.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.933+0000 7f1181092640 1 --2- 172.21.15.69:0/2047664071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f117c06c5c0 0x7f117c1581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:47.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.933+0000 7f1181092640 1 -- 172.21.15.69:0/2047664071 >> 172.21.15.69:0/2047664071 conn(0x7f117c0d1ae0 msgr2=0x7f117c15f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:47.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.933+0000 7f1181092640 1 -- 172.21.15.69:0/2047664071 shutdown_connections 2024-04-25T11:00:47.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:47.933+0000 7f1181092640 1 -- 172.21.15.69:0/2047664071 wait complete. 2024-04-25T11:00:47.945 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2024-04-25T11:00:46.423151+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220220,"kb_used_data":4072,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754500,"statfs":{"total":773094113280,"available":772868608000,"internally_reserved":0,"allocated":4169728,"data_stored":2341406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16341,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-86,"num_read_kb":-88,"num_write":-74,"num_write_kb":-42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000883"},"pg_stats":[{"pgid":"1.0","version":"15'72","reported_seq":131,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T11:00:30.092907+0000","last_change":"2024-04-25T10:59:07.839466+0000","last_active":"2024-04-25T11:00:30.092907+0000","last_peered":"2024-04-25T11:00:30.092907+0000","last_clean":"2024-04-25T11:00:30.092907+0000","last_became_active":"2024-04-25T10:59:07.836269+0000","last_became_peered":"2024-04-25T10:59:07.836269+0000","last_unstale":"2024-04-25T11:00:30.092907+0000","last_undegraded":"2024-04-25T11:00:30.092907+0000","last_fullsized":"2024-04-25T11:00:30.092907+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:06.549663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:06.549663+0000","last_clean_scrub_stamp":"2024-04-25T10:59:06.549663+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:21.575330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":958464,"data_stored":730989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":141147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":141203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":140635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":393216,"data_stored":157966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":364544,"data_stored":140601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":393216,"data_stored":157966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":954368,"data_stored":730899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:00:47.945 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:00:48.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.052+0000 7f8570b72640 1 Processor -- start 2024-04-25T11:00:48.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.052+0000 7f8570b72640 1 -- start start 2024-04-25T11:00:48.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.053+0000 7f8570b72640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f856c0cfd40 0x7f856c0b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:48.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.053+0000 7f8570b72640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f856c0b3e00 0x7f856c0b7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:48.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.053+0000 7f8570b72640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f856c0b7d80 0x7f856c0d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:48.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.053+0000 7f8570b72640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f856c0d27e0 con 0x7f856c0cfd40 2024-04-25T11:00:48.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.053+0000 7f8570b72640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f856c0b8190 con 0x7f856c0b7d80 2024-04-25T11:00:48.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.053+0000 7f8570b72640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f856c0d1c10 con 0x7f856c0b3e00 2024-04-25T11:00:48.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.054+0000 7f856a59c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f856c0cfd40 0x7f856c0b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:48.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.054+0000 7f856a59c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f856c0cfd40 0x7f856c0b21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47812/0 (socket says 172.21.15.69:47812) 2024-04-25T11:00:48.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.054+0000 7f856a59c640 1 -- 172.21.15.69:0/50457300 learned_addr learned my addr 172.21.15.69:0/50457300 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:48.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.054+0000 7f8569d9b640 1 --2- 172.21.15.69:0/50457300 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f856c0b3e00 0x7f856c0b7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:48.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.054+0000 7f856ad9d640 1 --2- 172.21.15.69:0/50457300 >> v2:172.21.15.146:3300/0 conn(0x7f856c0b7d80 0x7f856c0d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:48.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.054+0000 7f856a59c640 1 -- 172.21.15.69:0/50457300 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f856c0b3e00 msgr2=0x7f856c0b7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:48.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.054+0000 7f856a59c640 1 --2- 172.21.15.69:0/50457300 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f856c0b3e00 0x7f856c0b7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:48.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.054+0000 7f856a59c640 1 -- 172.21.15.69:0/50457300 >> v2:172.21.15.146:3300/0 conn(0x7f856c0b7d80 msgr2=0x7f856c0d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:48.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.054+0000 7f856a59c640 1 --2- 172.21.15.69:0/50457300 >> v2:172.21.15.146:3300/0 conn(0x7f856c0b7d80 0x7f856c0d2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:48.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.054+0000 7f856a59c640 1 -- 172.21.15.69:0/50457300 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f856c0d1eb0 con 0x7f856c0cfd40 2024-04-25T11:00:48.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.055+0000 7f856a59c640 1 --2- 172.21.15.69:0/50457300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f856c0cfd40 0x7f856c0b21a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f856c0b59b0 tx=0x7f855c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27c58c5b961bf82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:48.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.055+0000 7f856959a640 1 -- 172.21.15.69:0/50457300 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c037070 con 0x7f856c0cfd40 2024-04-25T11:00:48.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.055+0000 7f856959a640 1 -- 172.21.15.69:0/50457300 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f855c032440 con 0x7f856c0cfd40 2024-04-25T11:00:48.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.055+0000 7f856959a640 1 -- 172.21.15.69:0/50457300 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c032810 con 0x7f856c0cfd40 2024-04-25T11:00:48.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.055+0000 7f8570b72640 1 -- 172.21.15.69:0/50457300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f856c0cfd40 msgr2=0x7f856c0b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:48.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.055+0000 7f8570b72640 1 --2- 172.21.15.69:0/50457300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f856c0cfd40 0x7f856c0b21a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f856c0b59b0 tx=0x7f855c02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:00:48.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.056+0000 7f8570b72640 1 -- 172.21.15.69:0/50457300 shutdown_connections 2024-04-25T11:00:48.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.056+0000 7f8570b72640 1 --2- 172.21.15.69:0/50457300 >> v2:172.21.15.146:3300/0 conn(0x7f856c0b7d80 0x7f856c0d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:48.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.056+0000 7f8570b72640 1 --2- 172.21.15.69:0/50457300 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f856c0b3e00 0x7f856c0b7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:48.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.056+0000 7f8570b72640 1 --2- 172.21.15.69:0/50457300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f856c0cfd40 0x7f856c0b21a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:48.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.056+0000 7f8570b72640 1 -- 172.21.15.69:0/50457300 >> 172.21.15.69:0/50457300 conn(0x7f856c09b7d0 msgr2=0x7f856c09bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:48.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.056+0000 7f8570b72640 1 -- 172.21.15.69:0/50457300 shutdown_connections 2024-04-25T11:00:48.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.056+0000 7f8570b72640 1 -- 172.21.15.69:0/50457300 wait complete. 2024-04-25T11:00:48.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.057+0000 7f8570b72640 1 Processor -- start 2024-04-25T11:00:48.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.057+0000 7f8570b72640 1 -- start start 2024-04-25T11:00:48.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.059+0000 7f8570b72640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f856c0b3e00 0x7f856c14acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:48.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.059+0000 7f8570b72640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f856c0b7d80 0x7f856c14d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:48.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.059+0000 7f8570b72640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f856c0cfd40 0x7f856c0cc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:48.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.059+0000 7f8570b72640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f856c0b4210 con 0x7f856c0b3e00 2024-04-25T11:00:48.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.059+0000 7f8570b72640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f856c0d2470 con 0x7f856c0cfd40 2024-04-25T11:00:48.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.059+0000 7f8570b72640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f856c0b4390 con 0x7f856c0b7d80 2024-04-25T11:00:48.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.059+0000 7f856a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f856c0b3e00 0x7f856c14acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:48.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.059+0000 7f856a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f856c0b3e00 0x7f856c14acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47822/0 (socket says 172.21.15.69:47822) 2024-04-25T11:00:48.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.059+0000 7f856a59c640 1 -- 172.21.15.69:0/1507832409 learned_addr learned my addr 172.21.15.69:0/1507832409 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:48.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.059+0000 7f8569d9b640 1 --2- 172.21.15.69:0/1507832409 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f856c0b7d80 0x7f856c14d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:48.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.059+0000 7f856ad9d640 1 --2- 172.21.15.69:0/1507832409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f856c0cfd40 0x7f856c0cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:48.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.059+0000 7f856a59c640 1 -- 172.21.15.69:0/1507832409 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f856c0b7d80 msgr2=0x7f856c14d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:48.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.059+0000 7f856a59c640 1 --2- 172.21.15.69:0/1507832409 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f856c0b7d80 0x7f856c14d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:48.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.059+0000 7f856a59c640 1 -- 172.21.15.69:0/1507832409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f856c0cfd40 msgr2=0x7f856c0cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:48.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.059+0000 7f856a59c640 1 --2- 172.21.15.69:0/1507832409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f856c0cfd40 0x7f856c0cc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:48.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.059+0000 7f856a59c640 1 -- 172.21.15.69:0/1507832409 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f856c14e120 con 0x7f856c0b3e00 2024-04-25T11:00:48.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.059+0000 7f856a59c640 1 --2- 172.21.15.69:0/1507832409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f856c0b3e00 0x7f856c14acf0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f855c02a220 tx=0x7f855c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:48.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.059+0000 7f855affd640 1 -- 172.21.15.69:0/1507832409 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c041020 con 0x7f856c0b3e00 2024-04-25T11:00:48.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.060+0000 7f855affd640 1 -- 172.21.15.69:0/1507832409 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f855c0306c0 con 0x7f856c0b3e00 2024-04-25T11:00:48.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.060+0000 7f8570b72640 1 -- 172.21.15.69:0/1507832409 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f856c160da0 con 0x7f856c0b3e00 2024-04-25T11:00:48.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.060+0000 7f855affd640 1 -- 172.21.15.69:0/1507832409 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c037040 con 0x7f856c0b3e00 2024-04-25T11:00:48.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.060+0000 7f8570b72640 1 -- 172.21.15.69:0/1507832409 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f856c161360 con 0x7f856c0b3e00 2024-04-25T11:00:48.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.060+0000 7f8570b72640 1 -- 172.21.15.69:0/1507832409 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f856c050740 con 0x7f856c0b3e00 2024-04-25T11:00:48.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.061+0000 7f855affd640 1 -- 172.21.15.69:0/1507832409 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f855c044020 con 0x7f856c0b3e00 2024-04-25T11:00:48.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.062+0000 7f855affd640 1 --2- 172.21.15.69:0/1507832409 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f854c0744b0 0x7f854c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:48.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.062+0000 7f855affd640 1 -- 172.21.15.69:0/1507832409 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7f855c0b5830 con 0x7f856c0b3e00 2024-04-25T11:00:48.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.062+0000 7f8569d9b640 1 --2- 172.21.15.69:0/1507832409 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f854c0744b0 0x7f854c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:48.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.062+0000 7f8569d9b640 1 --2- 172.21.15.69:0/1507832409 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f854c0744b0 0x7f854c0769c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8560006fd0 tx=0x7f8560008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:48.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.066+0000 7f855affd640 1 -- 172.21.15.69:0/1507832409 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f855c04b030 con 0x7f856c0b3e00 2024-04-25T11:00:48.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.243+0000 7f8570b72640 1 -- 172.21.15.69:0/1507832409 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f856c0c9640 con 0x7f856c0b3e00 2024-04-25T11:00:48.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.244+0000 7f855affd640 1 -- 172.21.15.69:0/1507832409 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+13150 (secure 0 0 0) 0x7f855c080260 con 0x7f856c0b3e00 2024-04-25T11:00:48.245 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:00:48.245 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":35,"fsid":"ed0c8fe4-1bbb-4705-be72-eeea5bef859e","created":"2024-04-25T10:58:54.282571+0000","modified":"2024-04-25T11:00:45.131183+0000","last_up_change":"2024-04-25T10:59:05.546839+0000","last_in_change":"2024-04-25T10:58:56.625383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:59:06.422674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c380208-30e4-47a1-83b0-423ae5ead112","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6825","nonce":1876101065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6827","nonce":1876101065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6831","nonce":1876101065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":1876101065},{"type":"v1","addr":"172.21.15.69:6829","nonce":1876101065}]},"public_addr":"172.21.15.69:6825/1876101065","cluster_addr":"172.21.15.69:6827/1876101065","heartbeat_back_addr":"172.21.15.69:6831/1876101065","heartbeat_front_addr":"172.21.15.69:6829/1876101065","state":["exists","up"]},{"osd":1,"uuid":"b187c42c-5042-4ecd-9da5-f6b00cb9348e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6801","nonce":1806438563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6803","nonce":1806438563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6807","nonce":1806438563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1806438563},{"type":"v1","addr":"172.21.15.69:6805","nonce":1806438563}]},"public_addr":"172.21.15.69:6801/1806438563","cluster_addr":"172.21.15.69:6803/1806438563","heartbeat_back_addr":"172.21.15.69:6807/1806438563","heartbeat_front_addr":"172.21.15.69:6805/1806438563","state":["exists","up"]},{"osd":2,"uuid":"7accb18d-5798-4aca-a73c-587a47c0d982","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6817","nonce":1593617983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6819","nonce":1593617983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6823","nonce":1593617983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1593617983},{"type":"v1","addr":"172.21.15.69:6821","nonce":1593617983}]},"public_addr":"172.21.15.69:6817/1593617983","cluster_addr":"172.21.15.69:6819/1593617983","heartbeat_back_addr":"172.21.15.69:6823/1593617983","heartbeat_front_addr":"172.21.15.69:6821/1593617983","state":["exists","up"]},{"osd":3,"uuid":"fd51970a-0370-46cc-afbe-ad075f94285d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6809","nonce":3950661294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6811","nonce":3950661294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6815","nonce":3950661294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3950661294},{"type":"v1","addr":"172.21.15.69:6813","nonce":3950661294}]},"public_addr":"172.21.15.69:6809/3950661294","cluster_addr":"172.21.15.69:6811/3950661294","heartbeat_back_addr":"172.21.15.69:6815/3950661294","heartbeat_front_addr":"172.21.15.69:6813/3950661294","state":["exists","up"]},{"osd":4,"uuid":"f4ad69ae-fef4-431d-bd99-d3eeada93c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6801","nonce":2508929872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6803","nonce":2508929872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6807","nonce":2508929872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":2508929872},{"type":"v1","addr":"172.21.15.146:6805","nonce":2508929872}]},"public_addr":"172.21.15.146:6801/2508929872","cluster_addr":"172.21.15.146:6803/2508929872","heartbeat_back_addr":"172.21.15.146:6807/2508929872","heartbeat_front_addr":"172.21.15.146:6805/2508929872","state":["exists","up"]},{"osd":5,"uuid":"c6370c19-9ca2-4590-92e0-2a174da6c834","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6811","nonce":1310911138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6816","nonce":1310911138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6823","nonce":1310911138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":1310911138},{"type":"v1","addr":"172.21.15.146:6821","nonce":1310911138}]},"public_addr":"172.21.15.146:6811/1310911138","cluster_addr":"172.21.15.146:6816/1310911138","heartbeat_back_addr":"172.21.15.146:6823/1310911138","heartbeat_front_addr":"172.21.15.146:6821/1310911138","state":["exists","up"]},{"osd":6,"uuid":"e0e6a832-b1a1-42d7-836d-b4906e9860a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6810","nonce":634688021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6813","nonce":634688021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6820","nonce":634688021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":634688021},{"type":"v1","addr":"172.21.15.146:6817","nonce":634688021}]},"public_addr":"172.21.15.146:6810/634688021","cluster_addr":"172.21.15.146:6813/634688021","heartbeat_back_addr":"172.21.15.146:6820/634688021","heartbeat_front_addr":"172.21.15.146:6817/634688021","state":["exists","up"]},{"osd":7,"uuid":"25c077b9-5d8a-47ec-a83f-c8799112d7d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6825","nonce":3814215935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6827","nonce":3814215935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6831","nonce":3814215935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":3814215935},{"type":"v1","addr":"172.21.15.146:6829","nonce":3814215935}]},"public_addr":"172.21.15.146:6825/3814215935","cluster_addr":"172.21.15.146:6827/3814215935","heartbeat_back_addr":"172.21.15.146:6831/3814215935","heartbeat_front_addr":"172.21.15.146:6829/3814215935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.820200+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.850189+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.764560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.866248+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.807897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.869881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.782310+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:59:01.827131+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:6836/3766456547":"2024-04-26T11:00:41.915747+0000","172.21.15.69:6834/3766456547":"2024-04-26T11:00:41.915747+0000","172.21.15.69:6837/3153221129":"2024-04-26T11:00:30.066863+0000","172.21.15.69:6835/3153221129":"2024-04-26T11:00:30.066863+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:00:48.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.246+0000 7f8570b72640 1 -- 172.21.15.69:0/1507832409 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f854c0744b0 msgr2=0x7f854c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:48.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.246+0000 7f8570b72640 1 --2- 172.21.15.69:0/1507832409 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f854c0744b0 0x7f854c0769c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8560006fd0 tx=0x7f8560008040 comp rx=0 tx=0).stop 2024-04-25T11:00:48.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.246+0000 7f8570b72640 1 -- 172.21.15.69:0/1507832409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f856c0b3e00 msgr2=0x7f856c14acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:48.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.246+0000 7f8570b72640 1 --2- 172.21.15.69:0/1507832409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f856c0b3e00 0x7f856c14acf0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f855c02a220 tx=0x7f855c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:00:48.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.246+0000 7f8570b72640 1 -- 172.21.15.69:0/1507832409 shutdown_connections 2024-04-25T11:00:48.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.246+0000 7f8570b72640 1 --2- 172.21.15.69:0/1507832409 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f854c0744b0 0x7f854c0769c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:48.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.246+0000 7f8570b72640 1 --2- 172.21.15.69:0/1507832409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f856c0cfd40 0x7f856c0cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:48.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.246+0000 7f8570b72640 1 --2- 172.21.15.69:0/1507832409 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f856c0b7d80 0x7f856c14d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:48.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.246+0000 7f8570b72640 1 --2- 172.21.15.69:0/1507832409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f856c0b3e00 0x7f856c14acf0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:48.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.246+0000 7f8570b72640 1 -- 172.21.15.69:0/1507832409 >> 172.21.15.69:0/1507832409 conn(0x7f856c09b7d0 msgr2=0x7f856c071a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:48.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.246+0000 7f8570b72640 1 -- 172.21.15.69:0/1507832409 shutdown_connections 2024-04-25T11:00:48.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:48.246+0000 7f8570b72640 1 -- 172.21.15.69:0/1507832409 wait complete. 2024-04-25T11:00:48.314 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:00:48.317 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:00:49.259 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T11:00:49.259 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:00:49.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.384+0000 7f588e914640 1 Processor -- start 2024-04-25T11:00:49.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.384+0000 7f588e914640 1 -- start start 2024-04-25T11:00:49.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.386+0000 7f588e914640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58880ce250 0x7f588815cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:49.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.386+0000 7f588e914640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f58881458c0 0x7f58880b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:49.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.386+0000 7f588e914640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5888145cd0 con 0x7f588806d5d0 2024-04-25T11:00:49.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.386+0000 7f588e914640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f588815d5c0 con 0x7f58881458c0 2024-04-25T11:00:49.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.386+0000 7f588e914640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f588815d8a0 con 0x7f58880ce250 2024-04-25T11:00:49.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.386+0000 7f58877fe640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f58881458c0 0x7f58880b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:49.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.386+0000 7f58877fe640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f58881458c0 0x7f58880b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47832/0 (socket says 172.21.15.69:47832) 2024-04-25T11:00:49.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.386+0000 7f58877fe640 1 -- 172.21.15.69:0/1805888742 learned_addr learned my addr 172.21.15.69:0/1805888742 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:49.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.386+0000 7f5887fff640 1 --2- 172.21.15.69:0/1805888742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58880ce250 0x7f588815cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:49.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.386+0000 7f58877fe640 1 -- 172.21.15.69:0/1805888742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58880ce250 msgr2=0x7f588815cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:49.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.386+0000 7f58877fe640 1 --2- 172.21.15.69:0/1805888742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58880ce250 0x7f588815cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:49.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.386+0000 7f58877fe640 1 -- 172.21.15.69:0/1805888742 >> v1:172.21.15.146:6789/0 conn(0x7f588806d5d0 legacy=0x7f588806d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:49.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.386+0000 7f58877fe640 1 -- 172.21.15.69:0/1805888742 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588815db80 con 0x7f58881458c0 2024-04-25T11:00:49.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.387+0000 7f58877fe640 1 --2- 172.21.15.69:0/1805888742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58881458c0 0x7f58880b47b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f587c009870 tx=0x7f587c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=734ab0ae8c021c72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:49.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.387+0000 7f5886ffd640 1 -- 172.21.15.69:0/1805888742 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c037070 con 0x7f58881458c0 2024-04-25T11:00:49.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.387+0000 7f5886ffd640 1 -- 172.21.15.69:0/1805888742 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f587c032440 con 0x7f58881458c0 2024-04-25T11:00:49.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.387+0000 7f5886ffd640 1 -- 172.21.15.69:0/1805888742 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c032810 con 0x7f58881458c0 2024-04-25T11:00:49.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.388+0000 7f588e914640 1 -- 172.21.15.69:0/1805888742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58881458c0 msgr2=0x7f58880b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:49.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.388+0000 7f588e914640 1 --2- 172.21.15.69:0/1805888742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58881458c0 0x7f58880b47b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f587c009870 tx=0x7f587c02a1c0 comp rx=0 tx=0).stop 2024-04-25T11:00:49.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.388+0000 7f588e914640 1 -- 172.21.15.69:0/1805888742 shutdown_connections 2024-04-25T11:00:49.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.388+0000 7f588e914640 1 --2- 172.21.15.69:0/1805888742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58881458c0 0x7f58880b47b0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:49.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.388+0000 7f588e914640 1 --2- 172.21.15.69:0/1805888742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58880ce250 0x7f588815cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:49.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.388+0000 7f588e914640 1 -- 172.21.15.69:0/1805888742 >> 172.21.15.69:0/1805888742 conn(0x7f5888073090 msgr2=0x7f58880734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:49.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.388+0000 7f588e914640 1 -- 172.21.15.69:0/1805888742 shutdown_connections 2024-04-25T11:00:49.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.388+0000 7f588e914640 1 -- 172.21.15.69:0/1805888742 wait complete. 2024-04-25T11:00:49.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.390+0000 7f588e914640 1 Processor -- start 2024-04-25T11:00:49.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.390+0000 7f588e914640 1 -- start start 2024-04-25T11:00:49.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.391+0000 7f588e914640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f588806d5d0 0x7f588815be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:49.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.391+0000 7f588e914640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58880ce250 0x7f5888139b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:49.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.391+0000 7f588e914640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58881458c0 0x7f58880aaa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:49.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.391+0000 7f588e914640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5888071310 con 0x7f588806d5d0 2024-04-25T11:00:49.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.391+0000 7f588e914640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f588815de60 con 0x7f58881458c0 2024-04-25T11:00:49.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.391+0000 7f588e914640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f58880b4cf0 con 0x7f58880ce250 2024-04-25T11:00:49.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.391+0000 7f5887fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f588806d5d0 0x7f588815be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:49.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.391+0000 7f5887fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f588806d5d0 0x7f588815be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47850/0 (socket says 172.21.15.69:47850) 2024-04-25T11:00:49.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.391+0000 7f5887fff640 1 -- 172.21.15.69:0/2510019100 learned_addr learned my addr 172.21.15.69:0/2510019100 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:49.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.392+0000 7f58877fe640 1 --2- 172.21.15.69:0/2510019100 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58880ce250 0x7f5888139b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:49.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.392+0000 7f588ceb1640 1 --2- 172.21.15.69:0/2510019100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58881458c0 0x7f58880aaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:49.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.392+0000 7f5887fff640 1 -- 172.21.15.69:0/2510019100 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58880ce250 msgr2=0x7f5888139b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:49.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.392+0000 7f5887fff640 1 --2- 172.21.15.69:0/2510019100 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58880ce250 0x7f5888139b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:49.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.392+0000 7f5887fff640 1 -- 172.21.15.69:0/2510019100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58881458c0 msgr2=0x7f58880aaa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:49.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.392+0000 7f5887fff640 1 --2- 172.21.15.69:0/2510019100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58881458c0 0x7f58880aaa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:49.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.392+0000 7f5887fff640 1 -- 172.21.15.69:0/2510019100 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5888137130 con 0x7f588806d5d0 2024-04-25T11:00:49.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.392+0000 7f5887fff640 1 --2- 172.21.15.69:0/2510019100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f588806d5d0 0x7f588815be60 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f587800d5d0 tx=0x7f587800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:49.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.392+0000 7f5884ff9640 1 -- 172.21.15.69:0/2510019100 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5878004240 con 0x7f588806d5d0 2024-04-25T11:00:49.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.392+0000 7f5884ff9640 1 -- 172.21.15.69:0/2510019100 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58780043e0 con 0x7f588806d5d0 2024-04-25T11:00:49.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.392+0000 7f588e914640 1 -- 172.21.15.69:0/2510019100 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5888131b00 con 0x7f588806d5d0 2024-04-25T11:00:49.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.392+0000 7f5884ff9640 1 -- 172.21.15.69:0/2510019100 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5878003830 con 0x7f588806d5d0 2024-04-25T11:00:49.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.392+0000 7f588e914640 1 -- 172.21.15.69:0/2510019100 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58881320c0 con 0x7f588806d5d0 2024-04-25T11:00:49.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.393+0000 7f588e914640 1 -- 172.21.15.69:0/2510019100 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5888050740 con 0x7f588806d5d0 2024-04-25T11:00:49.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.394+0000 7f5884ff9640 1 -- 172.21.15.69:0/2510019100 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f58780039d0 con 0x7f588806d5d0 2024-04-25T11:00:49.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.394+0000 7f5884ff9640 1 --2- 172.21.15.69:0/2510019100 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f58740744b0 0x7f58740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:49.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.394+0000 7f5884ff9640 1 -- 172.21.15.69:0/2510019100 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7f58780950a0 con 0x7f588806d5d0 2024-04-25T11:00:49.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.394+0000 7f58877fe640 1 --2- 172.21.15.69:0/2510019100 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f58740744b0 0x7f58740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:49.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.394+0000 7f5884ff9640 1 --2- 172.21.15.69:0/2510019100 >> [v2:172.21.15.69:6824/1876101065,v1:172.21.15.69:6825/1876101065] conn(0x7f587407a460 0x7f587407c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:49.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.394+0000 7f5884ff9640 1 -- 172.21.15.69:0/2510019100 --> [v2:172.21.15.69:6824/1876101065,v1:172.21.15.69:6825/1876101065] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f587801ae70 con 0x7f587407a460 2024-04-25T11:00:49.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.394+0000 7f5884ff9640 1 -- 172.21.15.69:0/2510019100 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f587809c050 con 0x7f588806d5d0 2024-04-25T11:00:49.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.395+0000 7f588ceb1640 1 --2- 172.21.15.69:0/2510019100 >> [v2:172.21.15.69:6824/1876101065,v1:172.21.15.69:6825/1876101065] conn(0x7f587407a460 0x7f587407c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:49.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.395+0000 7f58877fe640 1 --2- 172.21.15.69:0/2510019100 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f58740744b0 0x7f58740769c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f587c034000 tx=0x7f587c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:49.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.395+0000 7f588ceb1640 1 --2- 172.21.15.69:0/2510019100 >> [v2:172.21.15.69:6824/1876101065,v1:172.21.15.69:6825/1876101065] conn(0x7f587407a460 0x7f587407c8b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:49.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.396+0000 7f5884ff9640 1 -- 172.21.15.69:0/2510019100 <== osd.0 v2:172.21.15.69:6824/1876101065 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f587801ae70 con 0x7f587407a460 2024-04-25T11:00:49.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.408+0000 7f588e914640 1 -- 172.21.15.69:0/2510019100 --> [v2:172.21.15.69:6824/1876101065,v1:172.21.15.69:6825/1876101065] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f588815c830 con 0x7f587407a460 2024-04-25T11:00:49.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.409+0000 7f5884ff9640 1 -- 172.21.15.69:0/2510019100 <== osd.0 v2:172.21.15.69:6824/1876101065 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f588815c830 con 0x7f587407a460 2024-04-25T11:00:49.409 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T11:00:49.409 INFO:teuthology.orchestra.run.smithi069.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:00:49.409 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T11:00:49.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.409+0000 7f588e914640 1 -- 172.21.15.69:0/2510019100 >> [v2:172.21.15.69:6824/1876101065,v1:172.21.15.69:6825/1876101065] conn(0x7f587407a460 msgr2=0x7f587407c8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:49.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.409+0000 7f588e914640 1 --2- 172.21.15.69:0/2510019100 >> [v2:172.21.15.69:6824/1876101065,v1:172.21.15.69:6825/1876101065] conn(0x7f587407a460 0x7f587407c8b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:49.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.409+0000 7f588e914640 1 -- 172.21.15.69:0/2510019100 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f58740744b0 msgr2=0x7f58740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:49.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.409+0000 7f588e914640 1 --2- 172.21.15.69:0/2510019100 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f58740744b0 0x7f58740769c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f587c034000 tx=0x7f587c02e000 comp rx=0 tx=0).stop 2024-04-25T11:00:49.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.409+0000 7f588e914640 1 -- 172.21.15.69:0/2510019100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f588806d5d0 msgr2=0x7f588815be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:49.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.409+0000 7f588e914640 1 --2- 172.21.15.69:0/2510019100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f588806d5d0 0x7f588815be60 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f587800d5d0 tx=0x7f587800da90 comp rx=0 tx=0).stop 2024-04-25T11:00:49.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.409+0000 7f588ceb1640 1 -- 172.21.15.69:0/2510019100 reap_dead start 2024-04-25T11:00:49.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.409+0000 7f588e914640 1 -- 172.21.15.69:0/2510019100 shutdown_connections 2024-04-25T11:00:49.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.409+0000 7f588e914640 1 -- 172.21.15.69:0/2510019100 >> 172.21.15.69:0/2510019100 conn(0x7f5888073090 msgr2=0x7f58880b4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:49.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.409+0000 7f588e914640 1 -- 172.21.15.69:0/2510019100 shutdown_connections 2024-04-25T11:00:49.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.409+0000 7f588e914640 1 -- 172.21.15.69:0/2510019100 wait complete. 2024-04-25T11:00:49.421 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T11:00:49.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.527+0000 7fee39b6c640 1 Processor -- start 2024-04-25T11:00:49.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.527+0000 7fee39b6c640 1 -- start start 2024-04-25T11:00:49.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.529+0000 7fee39b6c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fee340cf5e0 0x7fee3406d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:49.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.529+0000 7fee39b6c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee3406d640 0x7fee3406f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:49.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.529+0000 7fee39b6c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fee3406f5d0 0x7fee34161c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:49.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.529+0000 7fee39b6c640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fee3406fef0 con 0x7fee3406f5d0 2024-04-25T11:00:49.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.529+0000 7fee39b6c640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fee3406cc70 con 0x7fee340cf5e0 2024-04-25T11:00:49.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.529+0000 7fee39b6c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fee3406f9e0 con 0x7fee3406d640 2024-04-25T11:00:49.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.529+0000 7fee33fff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fee3406f5d0 0x7fee34161c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:49.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.529+0000 7fee33fff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fee3406f5d0 0x7fee34161c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47856/0 (socket says 172.21.15.69:47856) 2024-04-25T11:00:49.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.529+0000 7fee33fff640 1 -- 172.21.15.69:0/2982813844 learned_addr learned my addr 172.21.15.69:0/2982813844 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:49.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.529+0000 7fee32ffd640 1 --2- 172.21.15.69:0/2982813844 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee3406d640 0x7fee3406f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:49.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.529+0000 7fee337fe640 1 --2- 172.21.15.69:0/2982813844 >> v2:172.21.15.146:3300/0 conn(0x7fee340cf5e0 0x7fee3406d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:49.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.529+0000 7fee33fff640 1 -- 172.21.15.69:0/2982813844 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee3406d640 msgr2=0x7fee3406f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:49.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.529+0000 7fee33fff640 1 --2- 172.21.15.69:0/2982813844 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee3406d640 0x7fee3406f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:49.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.529+0000 7fee33fff640 1 -- 172.21.15.69:0/2982813844 >> v2:172.21.15.146:3300/0 conn(0x7fee340cf5e0 msgr2=0x7fee3406d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:49.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.529+0000 7fee33fff640 1 --2- 172.21.15.69:0/2982813844 >> v2:172.21.15.146:3300/0 conn(0x7fee340cf5e0 0x7fee3406d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:49.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.529+0000 7fee33fff640 1 -- 172.21.15.69:0/2982813844 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3406c660 con 0x7fee3406f5d0 2024-04-25T11:00:49.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.530+0000 7fee33fff640 1 --2- 172.21.15.69:0/2982813844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee3406f5d0 0x7fee34161c40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fee34072820 tx=0x7fee2c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a127de36fc3f1365 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:49.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.530+0000 7fee327fc640 1 -- 172.21.15.69:0/2982813844 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c036070 con 0x7fee3406f5d0 2024-04-25T11:00:49.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.530+0000 7fee327fc640 1 -- 172.21.15.69:0/2982813844 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee2c031600 con 0x7fee3406f5d0 2024-04-25T11:00:49.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.530+0000 7fee327fc640 1 -- 172.21.15.69:0/2982813844 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c0319d0 con 0x7fee3406f5d0 2024-04-25T11:00:49.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.531+0000 7fee39b6c640 1 -- 172.21.15.69:0/2982813844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee3406f5d0 msgr2=0x7fee34161c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:49.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.531+0000 7fee39b6c640 1 --2- 172.21.15.69:0/2982813844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee3406f5d0 0x7fee34161c40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fee34072820 tx=0x7fee2c02a610 comp rx=0 tx=0).stop 2024-04-25T11:00:49.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.531+0000 7fee39b6c640 1 -- 172.21.15.69:0/2982813844 shutdown_connections 2024-04-25T11:00:49.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.531+0000 7fee39b6c640 1 --2- 172.21.15.69:0/2982813844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee3406f5d0 0x7fee34161c40 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:49.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.531+0000 7fee39b6c640 1 --2- 172.21.15.69:0/2982813844 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee3406d640 0x7fee3406f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:49.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.531+0000 7fee39b6c640 1 --2- 172.21.15.69:0/2982813844 >> v2:172.21.15.146:3300/0 conn(0x7fee340cf5e0 0x7fee3406d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:49.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.531+0000 7fee39b6c640 1 -- 172.21.15.69:0/2982813844 >> 172.21.15.69:0/2982813844 conn(0x7fee340d1ae0 msgr2=0x7fee340d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:49.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.531+0000 7fee39b6c640 1 -- 172.21.15.69:0/2982813844 shutdown_connections 2024-04-25T11:00:49.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.531+0000 7fee39b6c640 1 -- 172.21.15.69:0/2982813844 wait complete. 2024-04-25T11:00:49.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.533+0000 7fee39b6c640 1 Processor -- start 2024-04-25T11:00:49.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.533+0000 7fee39b6c640 1 -- start start 2024-04-25T11:00:49.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.534+0000 7fee39b6c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee3406d640 0x7fee3415b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:49.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.534+0000 7fee39b6c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee3406f5d0 0x7fee340afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:49.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.534+0000 7fee39b6c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee340cf5e0 0x7fee340aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:49.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.534+0000 7fee39b6c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fee340cfea0 con 0x7fee3406f5d0 2024-04-25T11:00:49.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.534+0000 7fee39b6c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fee3406c900 con 0x7fee340cf5e0 2024-04-25T11:00:49.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.534+0000 7fee39b6c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fee340d0020 con 0x7fee3406d640 2024-04-25T11:00:49.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.534+0000 7fee32ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee3406f5d0 0x7fee340afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:49.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.534+0000 7fee32ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee3406f5d0 0x7fee340afe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47862/0 (socket says 172.21.15.69:47862) 2024-04-25T11:00:49.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.534+0000 7fee32ffd640 1 -- 172.21.15.69:0/2383263715 learned_addr learned my addr 172.21.15.69:0/2383263715 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:49.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.534+0000 7fee337fe640 1 --2- 172.21.15.69:0/2383263715 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee3406d640 0x7fee3415b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:49.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.534+0000 7fee33fff640 1 --2- 172.21.15.69:0/2383263715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee340cf5e0 0x7fee340aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:49.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.535+0000 7fee32ffd640 1 -- 172.21.15.69:0/2383263715 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee3406d640 msgr2=0x7fee3415b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:49.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.535+0000 7fee32ffd640 1 --2- 172.21.15.69:0/2383263715 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee3406d640 0x7fee3415b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:49.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.535+0000 7fee32ffd640 1 -- 172.21.15.69:0/2383263715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee340cf5e0 msgr2=0x7fee340aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:49.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.535+0000 7fee32ffd640 1 --2- 172.21.15.69:0/2383263715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee340cf5e0 0x7fee340aec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:49.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.535+0000 7fee32ffd640 1 -- 172.21.15.69:0/2383263715 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee340b0e10 con 0x7fee3406f5d0 2024-04-25T11:00:49.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.535+0000 7fee32ffd640 1 --2- 172.21.15.69:0/2383263715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee3406f5d0 0x7fee340afe30 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fee2800d800 tx=0x7fee2800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:49.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.535+0000 7fee1bfff640 1 -- 172.21.15.69:0/2383263715 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee28018030 con 0x7fee3406f5d0 2024-04-25T11:00:49.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.535+0000 7fee1bfff640 1 -- 172.21.15.69:0/2383263715 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee280096a0 con 0x7fee3406f5d0 2024-04-25T11:00:49.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.535+0000 7fee39b6c640 1 -- 172.21.15.69:0/2383263715 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee340b10f0 con 0x7fee3406f5d0 2024-04-25T11:00:49.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.535+0000 7fee1bfff640 1 -- 172.21.15.69:0/2383263715 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee28009870 con 0x7fee3406f5d0 2024-04-25T11:00:49.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.535+0000 7fee39b6c640 1 -- 172.21.15.69:0/2383263715 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee3415b200 con 0x7fee3406f5d0 2024-04-25T11:00:49.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.536+0000 7fee39b6c640 1 -- 172.21.15.69:0/2383263715 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee34070760 con 0x7fee3406f5d0 2024-04-25T11:00:49.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.537+0000 7fee1bfff640 1 -- 172.21.15.69:0/2383263715 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fee2801b070 con 0x7fee3406f5d0 2024-04-25T11:00:49.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.537+0000 7fee1bfff640 1 --2- 172.21.15.69:0/2383263715 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fee200744b0 0x7fee200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:49.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.537+0000 7fee337fe640 1 --2- 172.21.15.69:0/2383263715 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fee200744b0 0x7fee200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:49.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.537+0000 7fee1bfff640 1 -- 172.21.15.69:0/2383263715 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7fee280950c0 con 0x7fee3406f5d0 2024-04-25T11:00:49.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.538+0000 7fee337fe640 1 --2- 172.21.15.69:0/2383263715 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fee200744b0 0x7fee200769c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fee34147e40 tx=0x7fee24008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:49.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.542+0000 7fee1bfff640 1 -- 172.21.15.69:0/2383263715 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fee2805f530 con 0x7fee3406f5d0 2024-04-25T11:00:49.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.713+0000 7fee39b6c640 1 -- 172.21.15.69:0/2383263715 --> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fee3415acd0 con 0x7fee200744b0 2024-04-25T11:00:49.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.714+0000 7fee1bfff640 1 -- 172.21.15.69:0/2383263715 <== mgr.4107 v2:172.21.15.69:6832/1107421267 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fee3415acd0 con 0x7fee200744b0 2024-04-25T11:00:49.714 INFO:teuthology.orchestra.run.smithi069.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T11:00:49.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.715+0000 7fee39b6c640 1 -- 172.21.15.69:0/2383263715 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fee200744b0 msgr2=0x7fee200769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:49.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.715+0000 7fee39b6c640 1 --2- 172.21.15.69:0/2383263715 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fee200744b0 0x7fee200769c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fee34147e40 tx=0x7fee24008040 comp rx=0 tx=0).stop 2024-04-25T11:00:49.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.715+0000 7fee39b6c640 1 -- 172.21.15.69:0/2383263715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee3406f5d0 msgr2=0x7fee340afe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:49.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.715+0000 7fee39b6c640 1 --2- 172.21.15.69:0/2383263715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee3406f5d0 0x7fee340afe30 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fee2800d800 tx=0x7fee2800dcc0 comp rx=0 tx=0).stop 2024-04-25T11:00:49.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.716+0000 7fee39b6c640 1 -- 172.21.15.69:0/2383263715 shutdown_connections 2024-04-25T11:00:49.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.716+0000 7fee39b6c640 1 --2- 172.21.15.69:0/2383263715 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fee200744b0 0x7fee200769c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:49.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.716+0000 7fee39b6c640 1 --2- 172.21.15.69:0/2383263715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee340cf5e0 0x7fee340aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:49.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.716+0000 7fee39b6c640 1 --2- 172.21.15.69:0/2383263715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee3406f5d0 0x7fee340afe30 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:49.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.716+0000 7fee39b6c640 1 --2- 172.21.15.69:0/2383263715 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee3406d640 0x7fee3415b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:49.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.716+0000 7fee39b6c640 1 -- 172.21.15.69:0/2383263715 >> 172.21.15.69:0/2383263715 conn(0x7fee340d1ae0 msgr2=0x7fee34162520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:49.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.716+0000 7fee39b6c640 1 -- 172.21.15.69:0/2383263715 shutdown_connections 2024-04-25T11:00:49.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.716+0000 7fee39b6c640 1 -- 172.21.15.69:0/2383263715 wait complete. 2024-04-25T11:00:49.727 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T11:00:49.728 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:00:49.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.839+0000 7fc92ef7a640 1 Processor -- start 2024-04-25T11:00:49.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.839+0000 7fc92ef7a640 1 -- start start 2024-04-25T11:00:49.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.841+0000 7fc92ef7a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9280b9270 0x7fc9280d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:49.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.841+0000 7fc92ef7a640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fc928161ef0 0x7fc928162300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:49.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.841+0000 7fc92ef7a640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9280b0690 con 0x7fc928162840 2024-04-25T11:00:49.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.841+0000 7fc92ef7a640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fc9280cfab0 con 0x7fc928161ef0 2024-04-25T11:00:49.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.841+0000 7fc92ef7a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc9280cfd90 con 0x7fc9280b9270 2024-04-25T11:00:49.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.841+0000 7fc927fff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fc928161ef0 0x7fc928162300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:49.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.841+0000 7fc92d517640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fc928162840 0x7fc9280ce800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:54414/0 (socket says 172.21.15.69:54414) 2024-04-25T11:00:49.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.841+0000 7fc92d517640 1 -- 172.21.15.69:0/2272408123 learned_addr learned my addr 172.21.15.69:0/2272408123 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:49.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.841+0000 7fc92cd16640 1 --2- 172.21.15.69:0/2272408123 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9280b9270 0x7fc9280d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:49.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.841+0000 7fc927fff640 1 -- 172.21.15.69:0/2272408123 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9280b9270 msgr2=0x7fc9280d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:49.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.841+0000 7fc927fff640 1 --2- 172.21.15.69:0/2272408123 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9280b9270 0x7fc9280d15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:49.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.841+0000 7fc927fff640 1 -- 172.21.15.69:0/2272408123 >> v1:172.21.15.69:6789/0 conn(0x7fc928162840 legacy=0x7fc9280ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:49.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.841+0000 7fc9277fe640 1 -- 172.21.15.69:0/2272408123 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 896664392 0 0) 0x7fc9280b0690 con 0x7fc928162840 2024-04-25T11:00:49.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.841+0000 7fc927fff640 1 -- 172.21.15.69:0/2272408123 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc928162c50 con 0x7fc928161ef0 2024-04-25T11:00:49.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.842+0000 7fc927fff640 1 --2- 172.21.15.69:0/2272408123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc928161ef0 0x7fc928162300 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc91c0097a0 tx=0x7fc91c02a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc8d30626095c648 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:49.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.842+0000 7fc9277fe640 1 -- 172.21.15.69:0/2272408123 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91c037070 con 0x7fc928161ef0 2024-04-25T11:00:49.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.842+0000 7fc9277fe640 1 -- 172.21.15.69:0/2272408123 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc91c02adb0 con 0x7fc928161ef0 2024-04-25T11:00:49.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.842+0000 7fc9277fe640 1 -- 172.21.15.69:0/2272408123 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91c032660 con 0x7fc928161ef0 2024-04-25T11:00:49.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.843+0000 7fc92ef7a640 1 -- 172.21.15.69:0/2272408123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc928161ef0 msgr2=0x7fc928162300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:49.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.843+0000 7fc92ef7a640 1 --2- 172.21.15.69:0/2272408123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc928161ef0 0x7fc928162300 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc91c0097a0 tx=0x7fc91c02a060 comp rx=0 tx=0).stop 2024-04-25T11:00:49.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.843+0000 7fc92ef7a640 1 -- 172.21.15.69:0/2272408123 shutdown_connections 2024-04-25T11:00:49.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.843+0000 7fc92ef7a640 1 --2- 172.21.15.69:0/2272408123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc928161ef0 0x7fc928162300 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:49.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.843+0000 7fc92ef7a640 1 --2- 172.21.15.69:0/2272408123 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9280b9270 0x7fc9280d15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:49.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.843+0000 7fc92ef7a640 1 -- 172.21.15.69:0/2272408123 >> 172.21.15.69:0/2272408123 conn(0x7fc9280d3aa0 msgr2=0x7fc9280d3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:49.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.843+0000 7fc92ef7a640 1 -- 172.21.15.69:0/2272408123 shutdown_connections 2024-04-25T11:00:49.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.843+0000 7fc92ef7a640 1 -- 172.21.15.69:0/2272408123 wait complete. 2024-04-25T11:00:49.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.845+0000 7fc92ef7a640 1 Processor -- start 2024-04-25T11:00:49.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.845+0000 7fc92ef7a640 1 -- start start 2024-04-25T11:00:49.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.846+0000 7fc92ef7a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9280b9270 0x7fc9280c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:49.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.846+0000 7fc92ef7a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc928161ef0 0x7fc92806ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:49.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.846+0000 7fc92ef7a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc928162840 0x7fc92815cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:49.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.846+0000 7fc92ef7a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc9280cef10 con 0x7fc9280b9270 2024-04-25T11:00:49.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.846+0000 7fc92ef7a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc9280b2110 con 0x7fc928161ef0 2024-04-25T11:00:49.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.846+0000 7fc92ef7a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc9280cf090 con 0x7fc928162840 2024-04-25T11:00:49.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.846+0000 7fc92cd16640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9280b9270 0x7fc9280c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:49.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.846+0000 7fc92cd16640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9280b9270 0x7fc9280c66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47882/0 (socket says 172.21.15.69:47882) 2024-04-25T11:00:49.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.846+0000 7fc92cd16640 1 -- 172.21.15.69:0/3588895576 learned_addr learned my addr 172.21.15.69:0/3588895576 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:49.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.846+0000 7fc927fff640 1 --2- 172.21.15.69:0/3588895576 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc928161ef0 0x7fc92806ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:49.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.846+0000 7fc92d517640 1 --2- 172.21.15.69:0/3588895576 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc928162840 0x7fc92815cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:49.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.847+0000 7fc92cd16640 1 -- 172.21.15.69:0/3588895576 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc928162840 msgr2=0x7fc92815cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:49.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.847+0000 7fc92cd16640 1 --2- 172.21.15.69:0/3588895576 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc928162840 0x7fc92815cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:49.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.847+0000 7fc92cd16640 1 -- 172.21.15.69:0/3588895576 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc928161ef0 msgr2=0x7fc92806ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:49.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.847+0000 7fc92cd16640 1 --2- 172.21.15.69:0/3588895576 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc928161ef0 0x7fc92806ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:49.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.847+0000 7fc92cd16640 1 -- 172.21.15.69:0/3588895576 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc92806daf0 con 0x7fc9280b9270 2024-04-25T11:00:49.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.847+0000 7fc92cd16640 1 --2- 172.21.15.69:0/3588895576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9280b9270 0x7fc9280c66e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc91800e870 tx=0x7fc91800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:49.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.847+0000 7fc9257fa640 1 -- 172.21.15.69:0/3588895576 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc918017070 con 0x7fc9280b9270 2024-04-25T11:00:49.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.847+0000 7fc9257fa640 1 -- 172.21.15.69:0/3588895576 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc91800ee60 con 0x7fc9280b9270 2024-04-25T11:00:49.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.847+0000 7fc92ef7a640 1 -- 172.21.15.69:0/3588895576 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc92806ddd0 con 0x7fc9280b9270 2024-04-25T11:00:49.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.847+0000 7fc9257fa640 1 -- 172.21.15.69:0/3588895576 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91800cb00 con 0x7fc9280b9270 2024-04-25T11:00:49.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.847+0000 7fc92ef7a640 1 -- 172.21.15.69:0/3588895576 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc92806e390 con 0x7fc9280b9270 2024-04-25T11:00:49.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.848+0000 7fc92ef7a640 1 -- 172.21.15.69:0/3588895576 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc928050740 con 0x7fc9280b9270 2024-04-25T11:00:49.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.848+0000 7fc9257fa640 1 -- 172.21.15.69:0/3588895576 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc91801a070 con 0x7fc9280b9270 2024-04-25T11:00:49.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.849+0000 7fc9257fa640 1 --2- 172.21.15.69:0/3588895576 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc9140744e0 0x7fc9140769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:49.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.849+0000 7fc927fff640 1 --2- 172.21.15.69:0/3588895576 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc9140744e0 0x7fc9140769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:49.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.849+0000 7fc9257fa640 1 -- 172.21.15.69:0/3588895576 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7fc9180950a0 con 0x7fc9280b9270 2024-04-25T11:00:49.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.849+0000 7fc9257fa640 1 --2- 172.21.15.69:0/3588895576 >> [v2:172.21.15.69:6800/1806438563,v1:172.21.15.69:6801/1806438563] conn(0x7fc91407a490 0x7fc91407c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:49.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.849+0000 7fc9257fa640 1 -- 172.21.15.69:0/3588895576 --> [v2:172.21.15.69:6800/1806438563,v1:172.21.15.69:6801/1806438563] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc918016ec0 con 0x7fc91407a490 2024-04-25T11:00:49.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.849+0000 7fc9257fa640 1 -- 172.21.15.69:0/3588895576 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc91809c050 con 0x7fc9280b9270 2024-04-25T11:00:49.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.849+0000 7fc92d517640 1 --2- 172.21.15.69:0/3588895576 >> [v2:172.21.15.69:6800/1806438563,v1:172.21.15.69:6801/1806438563] conn(0x7fc91407a490 0x7fc91407c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:49.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.849+0000 7fc927fff640 1 --2- 172.21.15.69:0/3588895576 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc9140744e0 0x7fc9140769f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc91c034000 tx=0x7fc91c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:49.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.850+0000 7fc92d517640 1 --2- 172.21.15.69:0/3588895576 >> [v2:172.21.15.69:6800/1806438563,v1:172.21.15.69:6801/1806438563] conn(0x7fc91407a490 0x7fc91407c8e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:49.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.850+0000 7fc9257fa640 1 -- 172.21.15.69:0/3588895576 <== osd.1 v2:172.21.15.69:6800/1806438563 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc918016ec0 con 0x7fc91407a490 2024-04-25T11:00:49.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.863+0000 7fc92ef7a640 1 -- 172.21.15.69:0/3588895576 --> [v2:172.21.15.69:6800/1806438563,v1:172.21.15.69:6801/1806438563] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc9280d4770 con 0x7fc91407a490 2024-04-25T11:00:49.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.863+0000 7fc9257fa640 1 -- 172.21.15.69:0/3588895576 <== osd.1 v2:172.21.15.69:6800/1806438563 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc9280d4770 con 0x7fc91407a490 2024-04-25T11:00:49.864 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T11:00:49.864 INFO:teuthology.orchestra.run.smithi069.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:00:49.864 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T11:00:49.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.864+0000 7fc92ef7a640 1 -- 172.21.15.69:0/3588895576 >> [v2:172.21.15.69:6800/1806438563,v1:172.21.15.69:6801/1806438563] conn(0x7fc91407a490 msgr2=0x7fc91407c8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:49.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.864+0000 7fc92ef7a640 1 --2- 172.21.15.69:0/3588895576 >> [v2:172.21.15.69:6800/1806438563,v1:172.21.15.69:6801/1806438563] conn(0x7fc91407a490 0x7fc91407c8e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:49.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.864+0000 7fc92ef7a640 1 -- 172.21.15.69:0/3588895576 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc9140744e0 msgr2=0x7fc9140769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:49.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.864+0000 7fc92ef7a640 1 --2- 172.21.15.69:0/3588895576 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc9140744e0 0x7fc9140769f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc91c034000 tx=0x7fc91c02e000 comp rx=0 tx=0).stop 2024-04-25T11:00:49.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.864+0000 7fc92ef7a640 1 -- 172.21.15.69:0/3588895576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9280b9270 msgr2=0x7fc9280c66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:49.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.864+0000 7fc92ef7a640 1 --2- 172.21.15.69:0/3588895576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9280b9270 0x7fc9280c66e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc91800e870 tx=0x7fc91800ed30 comp rx=0 tx=0).stop 2024-04-25T11:00:49.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.864+0000 7fc92d517640 1 -- 172.21.15.69:0/3588895576 reap_dead start 2024-04-25T11:00:49.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.864+0000 7fc92ef7a640 1 -- 172.21.15.69:0/3588895576 shutdown_connections 2024-04-25T11:00:49.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.864+0000 7fc92ef7a640 1 -- 172.21.15.69:0/3588895576 >> 172.21.15.69:0/3588895576 conn(0x7fc9280d3aa0 msgr2=0x7fc928071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:49.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.864+0000 7fc92ef7a640 1 -- 172.21.15.69:0/3588895576 shutdown_connections 2024-04-25T11:00:49.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.864+0000 7fc92ef7a640 1 -- 172.21.15.69:0/3588895576 wait complete. 2024-04-25T11:00:49.875 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T11:00:49.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.982+0000 7f5e31c70640 1 Processor -- start 2024-04-25T11:00:49.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.982+0000 7f5e31c70640 1 -- start start 2024-04-25T11:00:49.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.984+0000 7f5e31c70640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e2c0ce520 0x7f5e2c06c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:49.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.984+0000 7f5e31c70640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f5e2c15e320 0x7f5e2c0b7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:49.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.984+0000 7f5e31c70640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5e2c0b7d80 0x7f5e2c0d2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:49.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.984+0000 7f5e31c70640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f5e2c0ce930 con 0x7f5e2c15e320 2024-04-25T11:00:49.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.984+0000 7f5e31c70640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f5e2c0b8190 con 0x7f5e2c0b7d80 2024-04-25T11:00:49.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.984+0000 7f5e31c70640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5e2c0d0450 con 0x7f5e2c0ce520 2024-04-25T11:00:49.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.984+0000 7f5e2affd640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f5e2c15e320 0x7f5e2c0b7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:49.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.984+0000 7f5e2affd640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f5e2c15e320 0x7f5e2c0b7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47884/0 (socket says 172.21.15.69:47884) 2024-04-25T11:00:49.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.984+0000 7f5e2affd640 1 -- 172.21.15.69:0/1347694636 learned_addr learned my addr 172.21.15.69:0/1347694636 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:49.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.984+0000 7f5e2b7fe640 1 --2- 172.21.15.69:0/1347694636 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e2c0ce520 0x7f5e2c06c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:49.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.984+0000 7f5e2bfff640 1 --2- 172.21.15.69:0/1347694636 >> v2:172.21.15.146:3300/0 conn(0x7f5e2c0b7d80 0x7f5e2c0d2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:49.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.984+0000 7f5e2affd640 1 -- 172.21.15.69:0/1347694636 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e2c0ce520 msgr2=0x7f5e2c06c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:49.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.984+0000 7f5e2affd640 1 --2- 172.21.15.69:0/1347694636 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e2c0ce520 0x7f5e2c06c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:49.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.984+0000 7f5e2affd640 1 -- 172.21.15.69:0/1347694636 >> v2:172.21.15.146:3300/0 conn(0x7f5e2c0b7d80 msgr2=0x7f5e2c0d2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:49.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.984+0000 7f5e2affd640 1 --2- 172.21.15.69:0/1347694636 >> v2:172.21.15.146:3300/0 conn(0x7f5e2c0b7d80 0x7f5e2c0d2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:49.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.984+0000 7f5e2affd640 1 -- 172.21.15.69:0/1347694636 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e2c06cc50 con 0x7f5e2c15e320 2024-04-25T11:00:49.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.984+0000 7f5e2bfff640 1 --2- 172.21.15.69:0/1347694636 >> v2:172.21.15.146:3300/0 conn(0x7f5e2c0b7d80 0x7f5e2c0d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:49.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.985+0000 7f5e2affd640 1 --2- 172.21.15.69:0/1347694636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e2c15e320 0x7f5e2c0b7840 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c161e20 tx=0x7f5e2002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ce30329c45123f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:49.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.985+0000 7f5e2a7fc640 1 -- 172.21.15.69:0/1347694636 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e20037070 con 0x7f5e2c15e320 2024-04-25T11:00:49.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.985+0000 7f5e2a7fc640 1 -- 172.21.15.69:0/1347694636 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e20032440 con 0x7f5e2c15e320 2024-04-25T11:00:49.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.985+0000 7f5e2a7fc640 1 -- 172.21.15.69:0/1347694636 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e20032810 con 0x7f5e2c15e320 2024-04-25T11:00:49.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.986+0000 7f5e31c70640 1 -- 172.21.15.69:0/1347694636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e2c15e320 msgr2=0x7f5e2c0b7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:49.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.986+0000 7f5e31c70640 1 --2- 172.21.15.69:0/1347694636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e2c15e320 0x7f5e2c0b7840 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c161e20 tx=0x7f5e2002a140 comp rx=0 tx=0).stop 2024-04-25T11:00:49.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.986+0000 7f5e31c70640 1 -- 172.21.15.69:0/1347694636 shutdown_connections 2024-04-25T11:00:49.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.986+0000 7f5e31c70640 1 --2- 172.21.15.69:0/1347694636 >> v2:172.21.15.146:3300/0 conn(0x7f5e2c0b7d80 0x7f5e2c0d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:49.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.986+0000 7f5e31c70640 1 --2- 172.21.15.69:0/1347694636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e2c15e320 0x7f5e2c0b7840 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:49.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.986+0000 7f5e31c70640 1 --2- 172.21.15.69:0/1347694636 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e2c0ce520 0x7f5e2c06c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:49.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.986+0000 7f5e31c70640 1 -- 172.21.15.69:0/1347694636 >> 172.21.15.69:0/1347694636 conn(0x7f5e2c09b7d0 msgr2=0x7f5e2c09bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:49.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.986+0000 7f5e31c70640 1 -- 172.21.15.69:0/1347694636 shutdown_connections 2024-04-25T11:00:49.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.986+0000 7f5e31c70640 1 -- 172.21.15.69:0/1347694636 wait complete. 2024-04-25T11:00:49.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.988+0000 7f5e31c70640 1 Processor -- start 2024-04-25T11:00:49.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.988+0000 7f5e31c70640 1 -- start start 2024-04-25T11:00:49.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.989+0000 7f5e31c70640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e2c0b7d80 0x7f5e2c1f5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:49.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.989+0000 7f5e31c70640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e2c0ce520 0x7f5e2c14d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:49.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.989+0000 7f5e31c70640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e2c15e320 0x7f5e2c14fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:49.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.989+0000 7f5e31c70640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c06d4f0 con 0x7f5e2c0b7d80 2024-04-25T11:00:49.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.989+0000 7f5e31c70640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c06d210 con 0x7f5e2c0ce520 2024-04-25T11:00:49.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.989+0000 7f5e31c70640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f5e2c06d7d0 con 0x7f5e2c15e320 2024-04-25T11:00:49.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.989+0000 7f5e2b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e2c0b7d80 0x7f5e2c1f5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:49.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.989+0000 7f5e2b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e2c0b7d80 0x7f5e2c1f5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47892/0 (socket says 172.21.15.69:47892) 2024-04-25T11:00:49.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.989+0000 7f5e2b7fe640 1 -- 172.21.15.69:0/3983260320 learned_addr learned my addr 172.21.15.69:0/3983260320 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:49.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.989+0000 7f5e2b7fe640 1 -- 172.21.15.69:0/3983260320 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e2c15e320 msgr2=0x7f5e2c14fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:49.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.989+0000 7f5e2bfff640 1 --2- 172.21.15.69:0/3983260320 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e2c15e320 0x7f5e2c14fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:49.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.989+0000 7f5e2b7fe640 1 --2- 172.21.15.69:0/3983260320 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e2c15e320 0x7f5e2c14fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:49.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.989+0000 7f5e2b7fe640 1 -- 172.21.15.69:0/3983260320 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e2c0ce520 msgr2=0x7f5e2c14d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:49.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.989+0000 7f5e2affd640 1 --2- 172.21.15.69:0/3983260320 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e2c0ce520 0x7f5e2c14d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:49.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.990+0000 7f5e2b7fe640 1 --2- 172.21.15.69:0/3983260320 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e2c0ce520 0x7f5e2c14d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:49.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.990+0000 7f5e2b7fe640 1 -- 172.21.15.69:0/3983260320 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e2c1f8570 con 0x7f5e2c0b7d80 2024-04-25T11:00:49.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.990+0000 7f5e2b7fe640 1 --2- 172.21.15.69:0/3983260320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e2c0b7d80 0x7f5e2c1f5390 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00e840 tx=0x7f5e1c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:49.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.990+0000 7f5e13fff640 1 -- 172.21.15.69:0/3983260320 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c017070 con 0x7f5e2c0b7d80 2024-04-25T11:00:49.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.990+0000 7f5e13fff640 1 -- 172.21.15.69:0/3983260320 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e1c00c940 con 0x7f5e2c0b7d80 2024-04-25T11:00:49.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.990+0000 7f5e31c70640 1 -- 172.21.15.69:0/3983260320 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e2c1f8850 con 0x7f5e2c0b7d80 2024-04-25T11:00:49.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.990+0000 7f5e13fff640 1 -- 172.21.15.69:0/3983260320 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c00cd10 con 0x7f5e2c0b7d80 2024-04-25T11:00:49.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.990+0000 7f5e31c70640 1 -- 172.21.15.69:0/3983260320 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e2c14f5e0 con 0x7f5e2c0b7d80 2024-04-25T11:00:49.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.991+0000 7f5e31c70640 1 -- 172.21.15.69:0/3983260320 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e2c068c90 con 0x7f5e2c0b7d80 2024-04-25T11:00:49.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.992+0000 7f5e13fff640 1 -- 172.21.15.69:0/3983260320 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5e1c01a070 con 0x7f5e2c0b7d80 2024-04-25T11:00:49.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.992+0000 7f5e13fff640 1 --2- 172.21.15.69:0/3983260320 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5e180744b0 0x7f5e180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:49.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.992+0000 7f5e13fff640 1 -- 172.21.15.69:0/3983260320 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7f5e1c094e60 con 0x7f5e2c0b7d80 2024-04-25T11:00:49.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.992+0000 7f5e2affd640 1 --2- 172.21.15.69:0/3983260320 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5e180744b0 0x7f5e180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:49.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.993+0000 7f5e2affd640 1 --2- 172.21.15.69:0/3983260320 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5e180744b0 0x7f5e180769c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5e20034000 tx=0x7f5e2002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:49.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:49.997+0000 7f5e13fff640 1 -- 172.21.15.69:0/3983260320 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5e2c068c90 con 0x7f5e2c0b7d80 2024-04-25T11:00:50.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.174+0000 7f5e31c70640 1 -- 172.21.15.69:0/3983260320 --> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f5e2c050740 con 0x7f5e180744b0 2024-04-25T11:00:50.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.174+0000 7f5e13fff640 1 -- 172.21.15.69:0/3983260320 <== mgr.4107 v2:172.21.15.69:6832/1107421267 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5e2c050740 con 0x7f5e180744b0 2024-04-25T11:00:50.175 INFO:teuthology.orchestra.run.smithi069.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T11:00:50.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.176+0000 7f5e31c70640 1 -- 172.21.15.69:0/3983260320 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5e180744b0 msgr2=0x7f5e180769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.176+0000 7f5e31c70640 1 --2- 172.21.15.69:0/3983260320 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5e180744b0 0x7f5e180769c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5e20034000 tx=0x7f5e2002e000 comp rx=0 tx=0).stop 2024-04-25T11:00:50.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.176+0000 7f5e31c70640 1 -- 172.21.15.69:0/3983260320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e2c0b7d80 msgr2=0x7f5e2c1f5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.176+0000 7f5e31c70640 1 --2- 172.21.15.69:0/3983260320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e2c0b7d80 0x7f5e2c1f5390 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00e840 tx=0x7f5e1c00ed00 comp rx=0 tx=0).stop 2024-04-25T11:00:50.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.176+0000 7f5e31c70640 1 -- 172.21.15.69:0/3983260320 shutdown_connections 2024-04-25T11:00:50.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.176+0000 7f5e31c70640 1 --2- 172.21.15.69:0/3983260320 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f5e180744b0 0x7f5e180769c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.176+0000 7f5e31c70640 1 --2- 172.21.15.69:0/3983260320 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e2c15e320 0x7f5e2c14fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.176+0000 7f5e31c70640 1 --2- 172.21.15.69:0/3983260320 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e2c0ce520 0x7f5e2c14d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.176+0000 7f5e31c70640 1 --2- 172.21.15.69:0/3983260320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e2c0b7d80 0x7f5e2c1f5390 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.176+0000 7f5e31c70640 1 -- 172.21.15.69:0/3983260320 >> 172.21.15.69:0/3983260320 conn(0x7f5e2c09b7d0 msgr2=0x7f5e2c06ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:50.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.177+0000 7f5e31c70640 1 -- 172.21.15.69:0/3983260320 shutdown_connections 2024-04-25T11:00:50.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.177+0000 7f5e31c70640 1 -- 172.21.15.69:0/3983260320 wait complete. 2024-04-25T11:00:50.188 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T11:00:50.188 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:00:50.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.298+0000 7f680b746640 1 Processor -- start 2024-04-25T11:00:50.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.298+0000 7f680b746640 1 -- start start 2024-04-25T11:00:50.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.300+0000 7f680b746640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f68040b2ca0 0x7f68040b1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:50.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.300+0000 7f680b746640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68040d3170 con 0x7f6804164ad0 2024-04-25T11:00:50.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.300+0000 7f680b746640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68040d3450 con 0x7f6804164640 2024-04-25T11:00:50.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.300+0000 7f680b746640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f68040d2f40 con 0x7f68040b2ca0 2024-04-25T11:00:50.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.300+0000 7f6809ce3640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f68040b2ca0 0x7f68040b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:50.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.300+0000 7f6809ce3640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f68040b2ca0 0x7f68040b1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47896/0 (socket says 172.21.15.69:47896) 2024-04-25T11:00:50.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.300+0000 7f68094e2640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f6804164ad0 0x7f6804163f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:54416/0 (socket says 172.21.15.69:54416) 2024-04-25T11:00:50.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.300+0000 7f6809ce3640 1 -- 172.21.15.69:0/1722887654 learned_addr learned my addr 172.21.15.69:0/1722887654 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:50.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.300+0000 7f6809ce3640 1 -- 172.21.15.69:0/1722887654 >> v1:172.21.15.146:6789/0 conn(0x7f6804164640 legacy=0x7f68040b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.300+0000 7f6809ce3640 1 -- 172.21.15.69:0/1722887654 >> v1:172.21.15.69:6789/0 conn(0x7f6804164ad0 legacy=0x7f6804163f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.300+0000 7f67fbfff640 1 -- 172.21.15.69:0/1722887654 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1601488598 0 0) 0x7f68040d3170 con 0x7f6804164ad0 2024-04-25T11:00:50.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.300+0000 7f6809ce3640 1 -- 172.21.15.69:0/1722887654 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f680406d6d0 con 0x7f68040b2ca0 2024-04-25T11:00:50.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.301+0000 7f6809ce3640 1 --2- 172.21.15.69:0/1722887654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68040b2ca0 0x7f68040b1180 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009870 tx=0x7f67fc02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c050a598dc21ef2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:50.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.301+0000 7f67fbfff640 1 -- 172.21.15.69:0/1722887654 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc037070 con 0x7f68040b2ca0 2024-04-25T11:00:50.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.301+0000 7f67fbfff640 1 -- 172.21.15.69:0/1722887654 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67fc02ae50 con 0x7f68040b2ca0 2024-04-25T11:00:50.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.301+0000 7f67fbfff640 1 -- 172.21.15.69:0/1722887654 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc032860 con 0x7f68040b2ca0 2024-04-25T11:00:50.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.302+0000 7f680b746640 1 -- 172.21.15.69:0/1722887654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68040b2ca0 msgr2=0x7f68040b1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.302+0000 7f680b746640 1 --2- 172.21.15.69:0/1722887654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68040b2ca0 0x7f68040b1180 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009870 tx=0x7f67fc02a300 comp rx=0 tx=0).stop 2024-04-25T11:00:50.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.302+0000 7f680b746640 1 -- 172.21.15.69:0/1722887654 shutdown_connections 2024-04-25T11:00:50.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.302+0000 7f680b746640 1 --2- 172.21.15.69:0/1722887654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68040b2ca0 0x7f68040b1180 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.302+0000 7f680b746640 1 -- 172.21.15.69:0/1722887654 >> 172.21.15.69:0/1722887654 conn(0x7f68040d3aa0 msgr2=0x7f68040d3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:50.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.302+0000 7f680b746640 1 -- 172.21.15.69:0/1722887654 shutdown_connections 2024-04-25T11:00:50.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.302+0000 7f680b746640 1 -- 172.21.15.69:0/1722887654 wait complete. 2024-04-25T11:00:50.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.304+0000 7f680b746640 1 Processor -- start 2024-04-25T11:00:50.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.304+0000 7f680b746640 1 -- start start 2024-04-25T11:00:50.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.305+0000 7f680b746640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68040b2ca0 0x7f68040c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:50.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.305+0000 7f680b746640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6804164ad0 0x7f68040b68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:50.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.305+0000 7f680b746640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f68040b6e30 0x7f68040cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:50.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.305+0000 7f680b746640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f68040b30b0 con 0x7f68040b2ca0 2024-04-25T11:00:50.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.305+0000 7f680b746640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6804164ee0 con 0x7f6804164ad0 2024-04-25T11:00:50.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.305+0000 7f680b746640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f68040b9430 con 0x7f68040b6e30 2024-04-25T11:00:50.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.305+0000 7f68094e2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68040b2ca0 0x7f68040c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:50.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.305+0000 7f68094e2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68040b2ca0 0x7f68040c84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47898/0 (socket says 172.21.15.69:47898) 2024-04-25T11:00:50.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.305+0000 7f68094e2640 1 -- 172.21.15.69:0/3424928013 learned_addr learned my addr 172.21.15.69:0/3424928013 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:50.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.305+0000 7f6808ce1640 1 --2- 172.21.15.69:0/3424928013 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6804164ad0 0x7f68040b68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:50.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.305+0000 7f6809ce3640 1 --2- 172.21.15.69:0/3424928013 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f68040b6e30 0x7f68040cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:50.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.305+0000 7f68094e2640 1 -- 172.21.15.69:0/3424928013 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f68040b6e30 msgr2=0x7f68040cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.305+0000 7f68094e2640 1 --2- 172.21.15.69:0/3424928013 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f68040b6e30 0x7f68040cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.305+0000 7f68094e2640 1 -- 172.21.15.69:0/3424928013 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6804164ad0 msgr2=0x7f68040b68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.305+0000 7f68094e2640 1 --2- 172.21.15.69:0/3424928013 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6804164ad0 0x7f68040b68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.305+0000 7f68094e2640 1 -- 172.21.15.69:0/3424928013 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68040c76e0 con 0x7f68040b2ca0 2024-04-25T11:00:50.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.305+0000 7f6809ce3640 1 --2- 172.21.15.69:0/3424928013 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f68040b6e30 0x7f68040cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:00:50.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.306+0000 7f68094e2640 1 --2- 172.21.15.69:0/3424928013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68040b2ca0 0x7f68040c84a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f680000c710 tx=0x7f680000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:50.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.306+0000 7f67f9ffb640 1 -- 172.21.15.69:0/3424928013 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6800018070 con 0x7f68040b2ca0 2024-04-25T11:00:50.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.306+0000 7f67f9ffb640 1 -- 172.21.15.69:0/3424928013 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f680000ce20 con 0x7f68040b2ca0 2024-04-25T11:00:50.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.306+0000 7f67f9ffb640 1 -- 172.21.15.69:0/3424928013 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680000a860 con 0x7f68040b2ca0 2024-04-25T11:00:50.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.306+0000 7f680b746640 1 -- 172.21.15.69:0/3424928013 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68040c79c0 con 0x7f68040b2ca0 2024-04-25T11:00:50.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.306+0000 7f680b746640 1 -- 172.21.15.69:0/3424928013 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68040b8b00 con 0x7f68040b2ca0 2024-04-25T11:00:50.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.307+0000 7f680b746640 1 -- 172.21.15.69:0/3424928013 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f68040d4770 con 0x7f68040b2ca0 2024-04-25T11:00:50.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.307+0000 7f67f9ffb640 1 -- 172.21.15.69:0/3424928013 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f680001b070 con 0x7f68040b2ca0 2024-04-25T11:00:50.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.308+0000 7f67f9ffb640 1 --2- 172.21.15.69:0/3424928013 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f67e80744e0 0x7f67e80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:50.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.308+0000 7f67f9ffb640 1 -- 172.21.15.69:0/3424928013 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7f68000944a0 con 0x7f68040b2ca0 2024-04-25T11:00:50.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.308+0000 7f6808ce1640 1 --2- 172.21.15.69:0/3424928013 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f67e80744e0 0x7f67e80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:50.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.308+0000 7f67f9ffb640 1 --2- 172.21.15.69:0/3424928013 >> [v2:172.21.15.69:6816/1593617983,v1:172.21.15.69:6817/1593617983] conn(0x7f67e807a490 0x7f67e807c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:50.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.308+0000 7f67f9ffb640 1 -- 172.21.15.69:0/3424928013 --> [v2:172.21.15.69:6816/1593617983,v1:172.21.15.69:6817/1593617983] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f67e807cfb0 con 0x7f67e807a490 2024-04-25T11:00:50.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.308+0000 7f67f9ffb640 1 -- 172.21.15.69:0/3424928013 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f680009b050 con 0x7f68040b2ca0 2024-04-25T11:00:50.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.308+0000 7f6809ce3640 1 --2- 172.21.15.69:0/3424928013 >> [v2:172.21.15.69:6816/1593617983,v1:172.21.15.69:6817/1593617983] conn(0x7f67e807a490 0x7f67e807c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:50.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.308+0000 7f6808ce1640 1 --2- 172.21.15.69:0/3424928013 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f67e80744e0 0x7f67e80769f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6804160490 tx=0x7f67f4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:50.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.309+0000 7f6809ce3640 1 --2- 172.21.15.69:0/3424928013 >> [v2:172.21.15.69:6816/1593617983,v1:172.21.15.69:6817/1593617983] conn(0x7f67e807a490 0x7f67e807c8e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:50.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.309+0000 7f67f9ffb640 1 -- 172.21.15.69:0/3424928013 <== osd.2 v2:172.21.15.69:6816/1593617983 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f67e807cfb0 con 0x7f67e807a490 2024-04-25T11:00:50.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.322+0000 7f680b746640 1 -- 172.21.15.69:0/3424928013 --> [v2:172.21.15.69:6816/1593617983,v1:172.21.15.69:6817/1593617983] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f68040b72a0 con 0x7f67e807a490 2024-04-25T11:00:50.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.322+0000 7f67f9ffb640 1 -- 172.21.15.69:0/3424928013 <== osd.2 v2:172.21.15.69:6816/1593617983 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f68040b72a0 con 0x7f67e807a490 2024-04-25T11:00:50.323 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T11:00:50.323 INFO:teuthology.orchestra.run.smithi069.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:00:50.323 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T11:00:50.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.323+0000 7f680b746640 1 -- 172.21.15.69:0/3424928013 >> [v2:172.21.15.69:6816/1593617983,v1:172.21.15.69:6817/1593617983] conn(0x7f67e807a490 msgr2=0x7f67e807c8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.323+0000 7f680b746640 1 --2- 172.21.15.69:0/3424928013 >> [v2:172.21.15.69:6816/1593617983,v1:172.21.15.69:6817/1593617983] conn(0x7f67e807a490 0x7f67e807c8e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.323+0000 7f680b746640 1 -- 172.21.15.69:0/3424928013 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f67e80744e0 msgr2=0x7f67e80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.323+0000 7f680b746640 1 --2- 172.21.15.69:0/3424928013 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f67e80744e0 0x7f67e80769f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6804160490 tx=0x7f67f4009040 comp rx=0 tx=0).stop 2024-04-25T11:00:50.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.323+0000 7f680b746640 1 -- 172.21.15.69:0/3424928013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68040b2ca0 msgr2=0x7f68040c84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.323+0000 7f680b746640 1 --2- 172.21.15.69:0/3424928013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68040b2ca0 0x7f68040c84a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f680000c710 tx=0x7f680000cbd0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.323+0000 7f6809ce3640 1 -- 172.21.15.69:0/3424928013 reap_dead start 2024-04-25T11:00:50.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.323+0000 7f680b746640 1 -- 172.21.15.69:0/3424928013 shutdown_connections 2024-04-25T11:00:50.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.323+0000 7f680b746640 1 -- 172.21.15.69:0/3424928013 >> 172.21.15.69:0/3424928013 conn(0x7f68040d3aa0 msgr2=0x7f68040cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:50.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.323+0000 7f680b746640 1 -- 172.21.15.69:0/3424928013 shutdown_connections 2024-04-25T11:00:50.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.323+0000 7f680b746640 1 -- 172.21.15.69:0/3424928013 wait complete. 2024-04-25T11:00:50.334 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T11:00:50.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.442+0000 7fbf92bb7640 1 Processor -- start 2024-04-25T11:00:50.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.442+0000 7fbf92bb7640 1 -- start start 2024-04-25T11:00:50.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.444+0000 7fbf92bb7640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fbf8c0d1620 0x7fbf8c0d1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:50.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.444+0000 7fbf92bb7640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbf8c163130 0x7fbf8c0b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:50.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.444+0000 7fbf92bb7640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf8c0b0770 con 0x7fbf8c0ce800 2024-04-25T11:00:50.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.444+0000 7fbf92bb7640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fbf8c0b10a0 con 0x7fbf8c0d1620 2024-04-25T11:00:50.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.444+0000 7fbf92bb7640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fbf8c0d2100 con 0x7fbf8c163130 2024-04-25T11:00:50.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.444+0000 7fbf8bfff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fbf8c0d1620 0x7fbf8c0d1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:50.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.444+0000 7fbf90953640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fbf8c0ce800 0x7fbf8c162a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:54430/0 (socket says 172.21.15.69:54430) 2024-04-25T11:00:50.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.444+0000 7fbf8bfff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fbf8c0d1620 0x7fbf8c0d1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47910/0 (socket says 172.21.15.69:47910) 2024-04-25T11:00:50.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.444+0000 7fbf90953640 1 -- 172.21.15.69:0/1703838709 learned_addr learned my addr 172.21.15.69:0/1703838709 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:50.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.444+0000 7fbf8bfff640 1 -- 172.21.15.69:0/1703838709 >> v2:172.21.15.146:3300/0 conn(0x7fbf8c163130 msgr2=0x7fbf8c0b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:50.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.444+0000 7fbf8bfff640 1 --2- 172.21.15.69:0/1703838709 >> v2:172.21.15.146:3300/0 conn(0x7fbf8c163130 0x7fbf8c0b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.444+0000 7fbf8bfff640 1 -- 172.21.15.69:0/1703838709 >> v1:172.21.15.69:6789/0 conn(0x7fbf8c0ce800 legacy=0x7fbf8c162a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.444+0000 7fbf8bfff640 1 -- 172.21.15.69:0/1703838709 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf8c0b0a50 con 0x7fbf8c0d1620 2024-04-25T11:00:50.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.445+0000 7fbf8bfff640 1 --2- 172.21.15.69:0/1703838709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf8c0d1620 0x7fbf8c0d1bc0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c08d070 tx=0x7fbf80029fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a3284808c7dc47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:50.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.445+0000 7fbf8b7fe640 1 -- 172.21.15.69:0/1703838709 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf80037070 con 0x7fbf8c0d1620 2024-04-25T11:00:50.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.445+0000 7fbf8b7fe640 1 -- 172.21.15.69:0/1703838709 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf8002ac40 con 0x7fbf8c0d1620 2024-04-25T11:00:50.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.445+0000 7fbf8b7fe640 1 -- 172.21.15.69:0/1703838709 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf80032440 con 0x7fbf8c0d1620 2024-04-25T11:00:50.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.446+0000 7fbf92bb7640 1 -- 172.21.15.69:0/1703838709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf8c0d1620 msgr2=0x7fbf8c0d1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.446+0000 7fbf92bb7640 1 --2- 172.21.15.69:0/1703838709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf8c0d1620 0x7fbf8c0d1bc0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c08d070 tx=0x7fbf80029fb0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.446+0000 7fbf92bb7640 1 -- 172.21.15.69:0/1703838709 shutdown_connections 2024-04-25T11:00:50.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.446+0000 7fbf92bb7640 1 --2- 172.21.15.69:0/1703838709 >> v2:172.21.15.146:3300/0 conn(0x7fbf8c163130 0x7fbf8c0b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.446+0000 7fbf92bb7640 1 --2- 172.21.15.69:0/1703838709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf8c0d1620 0x7fbf8c0d1bc0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.446+0000 7fbf92bb7640 1 -- 172.21.15.69:0/1703838709 >> 172.21.15.69:0/1703838709 conn(0x7fbf8c0d3a50 msgr2=0x7fbf8c0d3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:50.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.446+0000 7fbf92bb7640 1 -- 172.21.15.69:0/1703838709 shutdown_connections 2024-04-25T11:00:50.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.446+0000 7fbf92bb7640 1 -- 172.21.15.69:0/1703838709 wait complete. 2024-04-25T11:00:50.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.448+0000 7fbf92bb7640 1 Processor -- start 2024-04-25T11:00:50.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.448+0000 7fbf92bb7640 1 -- start start 2024-04-25T11:00:50.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.449+0000 7fbf92bb7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf8c0ce800 0x7fbf8c155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:50.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.449+0000 7fbf92bb7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf8c0d1620 0x7fbf8c0b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:50.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.449+0000 7fbf92bb7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbf8c163130 0x7fbf8c161980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:50.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.449+0000 7fbf92bb7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbf8c163540 con 0x7fbf8c0d1620 2024-04-25T11:00:50.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.449+0000 7fbf92bb7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbf8c0b51c0 con 0x7fbf8c0ce800 2024-04-25T11:00:50.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.449+0000 7fbf92bb7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fbf8c15a440 con 0x7fbf8c163130 2024-04-25T11:00:50.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.449+0000 7fbf8bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf8c0d1620 0x7fbf8c0b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:50.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.449+0000 7fbf8bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf8c0d1620 0x7fbf8c0b3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47916/0 (socket says 172.21.15.69:47916) 2024-04-25T11:00:50.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.449+0000 7fbf8bfff640 1 -- 172.21.15.69:0/2466776945 learned_addr learned my addr 172.21.15.69:0/2466776945 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:50.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.450+0000 7fbf90953640 1 --2- 172.21.15.69:0/2466776945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf8c0ce800 0x7fbf8c155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:50.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.450+0000 7fbf91154640 1 --2- 172.21.15.69:0/2466776945 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbf8c163130 0x7fbf8c161980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:50.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.450+0000 7fbf8bfff640 1 -- 172.21.15.69:0/2466776945 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbf8c163130 msgr2=0x7fbf8c161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.450+0000 7fbf8bfff640 1 --2- 172.21.15.69:0/2466776945 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbf8c163130 0x7fbf8c161980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.450+0000 7fbf8bfff640 1 -- 172.21.15.69:0/2466776945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf8c0ce800 msgr2=0x7fbf8c155f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.450+0000 7fbf8bfff640 1 --2- 172.21.15.69:0/2466776945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf8c0ce800 0x7fbf8c155f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.450+0000 7fbf8bfff640 1 -- 172.21.15.69:0/2466776945 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf8c0b4740 con 0x7fbf8c0d1620 2024-04-25T11:00:50.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.450+0000 7fbf90953640 1 --2- 172.21.15.69:0/2466776945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf8c0ce800 0x7fbf8c155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:50.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.450+0000 7fbf8bfff640 1 --2- 172.21.15.69:0/2466776945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf8c0d1620 0x7fbf8c0b3cd0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fbf8002a5a0 tx=0x7fbf800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:50.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.450+0000 7fbf897fa640 1 -- 172.21.15.69:0/2466776945 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf80037040 con 0x7fbf8c0d1620 2024-04-25T11:00:50.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.450+0000 7fbf897fa640 1 -- 172.21.15.69:0/2466776945 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf80004030 con 0x7fbf8c0d1620 2024-04-25T11:00:50.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.450+0000 7fbf897fa640 1 -- 172.21.15.69:0/2466776945 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf80004400 con 0x7fbf8c0d1620 2024-04-25T11:00:50.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.450+0000 7fbf92bb7640 1 -- 172.21.15.69:0/2466776945 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf8c0b2980 con 0x7fbf8c0d1620 2024-04-25T11:00:50.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.450+0000 7fbf92bb7640 1 -- 172.21.15.69:0/2466776945 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf8c0b2f00 con 0x7fbf8c0d1620 2024-04-25T11:00:50.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.451+0000 7fbf92bb7640 1 -- 172.21.15.69:0/2466776945 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf8c050740 con 0x7fbf8c0d1620 2024-04-25T11:00:50.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.452+0000 7fbf897fa640 1 -- 172.21.15.69:0/2466776945 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbf8003a070 con 0x7fbf8c0d1620 2024-04-25T11:00:50.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.452+0000 7fbf897fa640 1 --2- 172.21.15.69:0/2466776945 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fbf780744b0 0x7fbf780769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:50.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.452+0000 7fbf897fa640 1 -- 172.21.15.69:0/2466776945 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7fbf800b52e0 con 0x7fbf8c0d1620 2024-04-25T11:00:50.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.452+0000 7fbf90953640 1 --2- 172.21.15.69:0/2466776945 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fbf780744b0 0x7fbf780769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:50.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.453+0000 7fbf90953640 1 --2- 172.21.15.69:0/2466776945 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fbf780744b0 0x7fbf780769c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c001840 tx=0x7fbf7c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:50.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.457+0000 7fbf897fa640 1 -- 172.21.15.69:0/2466776945 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbf8c0b2980 con 0x7fbf8c0d1620 2024-04-25T11:00:50.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.630+0000 7fbf92bb7640 1 -- 172.21.15.69:0/2466776945 --> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fbf8c0d1220 con 0x7fbf780744b0 2024-04-25T11:00:50.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.631+0000 7fbf897fa640 1 -- 172.21.15.69:0/2466776945 <== mgr.4107 v2:172.21.15.69:6832/1107421267 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fbf8c0d1220 con 0x7fbf780744b0 2024-04-25T11:00:50.631 INFO:teuthology.orchestra.run.smithi069.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T11:00:50.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.632+0000 7fbf92bb7640 1 -- 172.21.15.69:0/2466776945 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fbf780744b0 msgr2=0x7fbf780769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.633+0000 7fbf92bb7640 1 --2- 172.21.15.69:0/2466776945 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fbf780744b0 0x7fbf780769c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c001840 tx=0x7fbf7c009040 comp rx=0 tx=0).stop 2024-04-25T11:00:50.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.633+0000 7fbf92bb7640 1 -- 172.21.15.69:0/2466776945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf8c0d1620 msgr2=0x7fbf8c0b3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.633+0000 7fbf92bb7640 1 --2- 172.21.15.69:0/2466776945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf8c0d1620 0x7fbf8c0b3cd0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fbf8002a5a0 tx=0x7fbf800046c0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.633+0000 7fbf92bb7640 1 -- 172.21.15.69:0/2466776945 shutdown_connections 2024-04-25T11:00:50.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.633+0000 7fbf92bb7640 1 --2- 172.21.15.69:0/2466776945 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fbf780744b0 0x7fbf780769c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.633+0000 7fbf92bb7640 1 --2- 172.21.15.69:0/2466776945 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbf8c163130 0x7fbf8c161980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.633+0000 7fbf92bb7640 1 --2- 172.21.15.69:0/2466776945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf8c0d1620 0x7fbf8c0b3cd0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.633+0000 7fbf92bb7640 1 --2- 172.21.15.69:0/2466776945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf8c0ce800 0x7fbf8c155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.633+0000 7fbf92bb7640 1 -- 172.21.15.69:0/2466776945 >> 172.21.15.69:0/2466776945 conn(0x7fbf8c0d3a50 msgr2=0x7fbf8c162a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:50.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.633+0000 7fbf92bb7640 1 -- 172.21.15.69:0/2466776945 shutdown_connections 2024-04-25T11:00:50.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.633+0000 7fbf92bb7640 1 -- 172.21.15.69:0/2466776945 wait complete. 2024-04-25T11:00:50.644 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T11:00:50.644 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:00:50.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.755+0000 7fb46994e640 1 Processor -- start 2024-04-25T11:00:50.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.755+0000 7fb46994e640 1 -- start start 2024-04-25T11:00:50.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.756+0000 7fb46994e640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb46408d630 0x7fb4640b90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:50.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.756+0000 7fb46994e640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb4640b1c50 0x7fb4640ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:50.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.756+0000 7fb46994e640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4640d21e0 con 0x7fb46408dec0 2024-04-25T11:00:50.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.756+0000 7fb46994e640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fb4640b95f0 con 0x7fb46408d630 2024-04-25T11:00:50.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.756+0000 7fb46994e640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fb4640d1940 con 0x7fb4640b1c50 2024-04-25T11:00:50.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.756+0000 7fb4627fc640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb46408d630 0x7fb4640b90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:50.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.756+0000 7fb4627fc640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fb46408d630 0x7fb4640b90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47922/0 (socket says 172.21.15.69:47922) 2024-04-25T11:00:50.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.756+0000 7fb462ffd640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fb46408dec0 0x7fb46408cf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:54436/0 (socket says 172.21.15.69:54436) 2024-04-25T11:00:50.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.756+0000 7fb4627fc640 1 -- 172.21.15.69:0/1908370801 learned_addr learned my addr 172.21.15.69:0/1908370801 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:50.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.757+0000 7fb4637fe640 1 --2- 172.21.15.69:0/1908370801 >> v2:172.21.15.146:3300/0 conn(0x7fb4640b1c50 0x7fb4640ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:50.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.757+0000 7fb4627fc640 1 -- 172.21.15.69:0/1908370801 >> v2:172.21.15.146:3300/0 conn(0x7fb4640b1c50 msgr2=0x7fb4640ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:50.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.757+0000 7fb4627fc640 1 --2- 172.21.15.69:0/1908370801 >> v2:172.21.15.146:3300/0 conn(0x7fb4640b1c50 0x7fb4640ce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.757+0000 7fb4627fc640 1 -- 172.21.15.69:0/1908370801 >> v1:172.21.15.69:6789/0 conn(0x7fb46408dec0 legacy=0x7fb46408cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.757+0000 7fb4627fc640 1 -- 172.21.15.69:0/1908370801 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4640d1c20 con 0x7fb46408d630 2024-04-25T11:00:50.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.757+0000 7fb4627fc640 1 --2- 172.21.15.69:0/1908370801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb46408d630 0x7fb4640b90b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb4640b9770 tx=0x7fb45802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3829a1ed09e5d26a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:50.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.757+0000 7fb461ffb640 1 -- 172.21.15.69:0/1908370801 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb458037070 con 0x7fb46408d630 2024-04-25T11:00:50.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.757+0000 7fb461ffb640 1 -- 172.21.15.69:0/1908370801 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb458032440 con 0x7fb46408d630 2024-04-25T11:00:50.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.757+0000 7fb461ffb640 1 -- 172.21.15.69:0/1908370801 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb458032810 con 0x7fb46408d630 2024-04-25T11:00:50.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.758+0000 7fb46994e640 1 -- 172.21.15.69:0/1908370801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb46408d630 msgr2=0x7fb4640b90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.758+0000 7fb46994e640 1 --2- 172.21.15.69:0/1908370801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb46408d630 0x7fb4640b90b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb4640b9770 tx=0x7fb45802a270 comp rx=0 tx=0).stop 2024-04-25T11:00:50.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.758+0000 7fb46994e640 1 -- 172.21.15.69:0/1908370801 shutdown_connections 2024-04-25T11:00:50.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.758+0000 7fb46994e640 1 --2- 172.21.15.69:0/1908370801 >> v2:172.21.15.146:3300/0 conn(0x7fb4640b1c50 0x7fb4640ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.758+0000 7fb46994e640 1 --2- 172.21.15.69:0/1908370801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb46408d630 0x7fb4640b90b0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.758+0000 7fb46994e640 1 -- 172.21.15.69:0/1908370801 >> 172.21.15.69:0/1908370801 conn(0x7fb4640d0510 msgr2=0x7fb4640d0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:50.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.758+0000 7fb46994e640 1 -- 172.21.15.69:0/1908370801 shutdown_connections 2024-04-25T11:00:50.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.758+0000 7fb46994e640 1 -- 172.21.15.69:0/1908370801 wait complete. 2024-04-25T11:00:50.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.760+0000 7fb46994e640 1 Processor -- start 2024-04-25T11:00:50.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.760+0000 7fb46994e640 1 -- start start 2024-04-25T11:00:50.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.761+0000 7fb46994e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb46408dec0 0x7fb46414d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:50.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.761+0000 7fb46994e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4640b1c50 0x7fb46414a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:50.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.761+0000 7fb46994e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb46415bd00 0x7fb46415d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:50.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.761+0000 7fb46994e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb4640d15d0 con 0x7fb4640b1c50 2024-04-25T11:00:50.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.761+0000 7fb46994e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb4640b0720 con 0x7fb46408dec0 2024-04-25T11:00:50.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.761+0000 7fb46994e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fb4640cedc0 con 0x7fb46415bd00 2024-04-25T11:00:50.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.761+0000 7fb4627fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4640b1c50 0x7fb46414a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:50.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.761+0000 7fb4627fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4640b1c50 0x7fb46414a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47924/0 (socket says 172.21.15.69:47924) 2024-04-25T11:00:50.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.761+0000 7fb4627fc640 1 -- 172.21.15.69:0/3028413919 learned_addr learned my addr 172.21.15.69:0/3028413919 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:50.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.762+0000 7fb4637fe640 1 --2- 172.21.15.69:0/3028413919 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb46415bd00 0x7fb46415d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:50.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.762+0000 7fb462ffd640 1 --2- 172.21.15.69:0/3028413919 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb46408dec0 0x7fb46414d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:50.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.762+0000 7fb4627fc640 1 -- 172.21.15.69:0/3028413919 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb46415bd00 msgr2=0x7fb46415d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.762+0000 7fb4627fc640 1 --2- 172.21.15.69:0/3028413919 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb46415bd00 0x7fb46415d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.762+0000 7fb4627fc640 1 -- 172.21.15.69:0/3028413919 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb46408dec0 msgr2=0x7fb46414d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.762+0000 7fb4627fc640 1 --2- 172.21.15.69:0/3028413919 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb46408dec0 0x7fb46414d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.762+0000 7fb4627fc640 1 -- 172.21.15.69:0/3028413919 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4640cd610 con 0x7fb4640b1c50 2024-04-25T11:00:50.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.762+0000 7fb4627fc640 1 --2- 172.21.15.69:0/3028413919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4640b1c50 0x7fb46414a2c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb458000c00 tx=0x7fb4580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:50.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.762+0000 7fb44bfff640 1 -- 172.21.15.69:0/3028413919 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb458037040 con 0x7fb4640b1c50 2024-04-25T11:00:50.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.762+0000 7fb44bfff640 1 -- 172.21.15.69:0/3028413919 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb458042070 con 0x7fb4640b1c50 2024-04-25T11:00:50.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.762+0000 7fb46994e640 1 -- 172.21.15.69:0/3028413919 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4640cd8f0 con 0x7fb4640b1c50 2024-04-25T11:00:50.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.762+0000 7fb44bfff640 1 -- 172.21.15.69:0/3028413919 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb458030bb0 con 0x7fb4640b1c50 2024-04-25T11:00:50.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.762+0000 7fb46994e640 1 -- 172.21.15.69:0/3028413919 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4641603e0 con 0x7fb4640b1c50 2024-04-25T11:00:50.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.763+0000 7fb46994e640 1 -- 172.21.15.69:0/3028413919 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb464050740 con 0x7fb4640b1c50 2024-04-25T11:00:50.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.764+0000 7fb44bfff640 1 -- 172.21.15.69:0/3028413919 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb458047460 con 0x7fb4640b1c50 2024-04-25T11:00:50.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.764+0000 7fb44bfff640 1 --2- 172.21.15.69:0/3028413919 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb4500744b0 0x7fb4500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:50.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.764+0000 7fb44bfff640 1 -- 172.21.15.69:0/3028413919 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7fb4580b6630 con 0x7fb4640b1c50 2024-04-25T11:00:50.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.764+0000 7fb462ffd640 1 --2- 172.21.15.69:0/3028413919 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb4500744b0 0x7fb4500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:50.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.764+0000 7fb44bfff640 1 --2- 172.21.15.69:0/3028413919 >> [v2:172.21.15.69:6808/3950661294,v1:172.21.15.69:6809/3950661294] conn(0x7fb45007a460 0x7fb45007c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:50.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.764+0000 7fb44bfff640 1 -- 172.21.15.69:0/3028413919 --> [v2:172.21.15.69:6808/3950661294,v1:172.21.15.69:6809/3950661294] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb458032440 con 0x7fb45007a460 2024-04-25T11:00:50.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.764+0000 7fb44bfff640 1 -- 172.21.15.69:0/3028413919 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb4580b6ac0 con 0x7fb4640b1c50 2024-04-25T11:00:50.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.765+0000 7fb4637fe640 1 --2- 172.21.15.69:0/3028413919 >> [v2:172.21.15.69:6808/3950661294,v1:172.21.15.69:6809/3950661294] conn(0x7fb45007a460 0x7fb45007c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:50.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.765+0000 7fb462ffd640 1 --2- 172.21.15.69:0/3028413919 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb4500744b0 0x7fb4500769c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb454000f60 tx=0x7fb454007ce0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:50.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.765+0000 7fb4637fe640 1 --2- 172.21.15.69:0/3028413919 >> [v2:172.21.15.69:6808/3950661294,v1:172.21.15.69:6809/3950661294] conn(0x7fb45007a460 0x7fb45007c8b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:50.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.766+0000 7fb44bfff640 1 -- 172.21.15.69:0/3028413919 <== osd.3 v2:172.21.15.69:6808/3950661294 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb458032440 con 0x7fb45007a460 2024-04-25T11:00:50.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.778+0000 7fb46994e640 1 -- 172.21.15.69:0/3028413919 --> [v2:172.21.15.69:6808/3950661294,v1:172.21.15.69:6809/3950661294] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb4640b22b0 con 0x7fb45007a460 2024-04-25T11:00:50.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.779+0000 7fb44bfff640 1 -- 172.21.15.69:0/3028413919 <== osd.3 v2:172.21.15.69:6808/3950661294 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb4640b22b0 con 0x7fb45007a460 2024-04-25T11:00:50.779 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T11:00:50.779 INFO:teuthology.orchestra.run.smithi069.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:00:50.779 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T11:00:50.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.779+0000 7fb46994e640 1 -- 172.21.15.69:0/3028413919 >> [v2:172.21.15.69:6808/3950661294,v1:172.21.15.69:6809/3950661294] conn(0x7fb45007a460 msgr2=0x7fb45007c8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.779+0000 7fb46994e640 1 --2- 172.21.15.69:0/3028413919 >> [v2:172.21.15.69:6808/3950661294,v1:172.21.15.69:6809/3950661294] conn(0x7fb45007a460 0x7fb45007c8b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.779+0000 7fb46994e640 1 -- 172.21.15.69:0/3028413919 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb4500744b0 msgr2=0x7fb4500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.779+0000 7fb46994e640 1 --2- 172.21.15.69:0/3028413919 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fb4500744b0 0x7fb4500769c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb454000f60 tx=0x7fb454007ce0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.779+0000 7fb46994e640 1 -- 172.21.15.69:0/3028413919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4640b1c50 msgr2=0x7fb46414a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.779+0000 7fb46994e640 1 --2- 172.21.15.69:0/3028413919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4640b1c50 0x7fb46414a2c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb458000c00 tx=0x7fb4580046c0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.779+0000 7fb4637fe640 1 -- 172.21.15.69:0/3028413919 reap_dead start 2024-04-25T11:00:50.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.779+0000 7fb46994e640 1 -- 172.21.15.69:0/3028413919 shutdown_connections 2024-04-25T11:00:50.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.779+0000 7fb46994e640 1 -- 172.21.15.69:0/3028413919 >> 172.21.15.69:0/3028413919 conn(0x7fb4640d0510 msgr2=0x7fb4640b0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:50.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.779+0000 7fb46994e640 1 -- 172.21.15.69:0/3028413919 shutdown_connections 2024-04-25T11:00:50.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.779+0000 7fb46994e640 1 -- 172.21.15.69:0/3028413919 wait complete. 2024-04-25T11:00:50.790 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T11:00:50.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.897+0000 7fea6ca0d640 1 Processor -- start 2024-04-25T11:00:50.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.897+0000 7fea6ca0d640 1 -- start start 2024-04-25T11:00:50.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.899+0000 7fea6ca0d640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fea680d4570 0x7fea680b4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:50.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.899+0000 7fea6ca0d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea6806e8f0 0x7fea680d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:50.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.899+0000 7fea6ca0d640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea6806d870 con 0x7fea680d09f0 2024-04-25T11:00:50.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.899+0000 7fea6ca0d640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fea6806e460 con 0x7fea680d4570 2024-04-25T11:00:50.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.899+0000 7fea6ca0d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fea6806db10 con 0x7fea6806e8f0 2024-04-25T11:00:50.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.899+0000 7fea6659c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fea680d4570 0x7fea680b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:50.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.899+0000 7fea6659c640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fea680d4570 0x7fea680b4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47938/0 (socket says 172.21.15.69:47938) 2024-04-25T11:00:50.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.899+0000 7fea66d9d640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fea680d09f0 0x7fea680b2dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:54450/0 (socket says 172.21.15.69:54450) 2024-04-25T11:00:50.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.899+0000 7fea6659c640 1 -- 172.21.15.69:0/2397022395 learned_addr learned my addr 172.21.15.69:0/2397022395 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:50.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.899+0000 7fea5dd9b640 1 --2- 172.21.15.69:0/2397022395 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea6806e8f0 0x7fea680d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:50.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.899+0000 7fea6659c640 1 -- 172.21.15.69:0/2397022395 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea6806e8f0 msgr2=0x7fea680d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:50.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.899+0000 7fea6659c640 1 --2- 172.21.15.69:0/2397022395 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea6806e8f0 0x7fea680d04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.899+0000 7fea6659c640 1 -- 172.21.15.69:0/2397022395 >> v1:172.21.15.69:6789/0 conn(0x7fea680d09f0 legacy=0x7fea680b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.899+0000 7fea65d9b640 1 -- 172.21.15.69:0/2397022395 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3134012049 0 0) 0x7fea6806d870 con 0x7fea680d09f0 2024-04-25T11:00:50.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.899+0000 7fea6659c640 1 -- 172.21.15.69:0/2397022395 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea6806e0f0 con 0x7fea680d4570 2024-04-25T11:00:50.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.900+0000 7fea6659c640 1 --2- 172.21.15.69:0/2397022395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea680d4570 0x7fea680b4830 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fea6806da40 tx=0x7fea5802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75791e576f8b4c89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:50.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.900+0000 7fea65d9b640 1 -- 172.21.15.69:0/2397022395 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea58004030 con 0x7fea680d4570 2024-04-25T11:00:50.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.900+0000 7fea65d9b640 1 -- 172.21.15.69:0/2397022395 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea580041d0 con 0x7fea680d4570 2024-04-25T11:00:50.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.900+0000 7fea65d9b640 1 -- 172.21.15.69:0/2397022395 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea58003610 con 0x7fea680d4570 2024-04-25T11:00:50.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.901+0000 7fea6ca0d640 1 -- 172.21.15.69:0/2397022395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea680d4570 msgr2=0x7fea680b4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.901+0000 7fea6ca0d640 1 --2- 172.21.15.69:0/2397022395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea680d4570 0x7fea680b4830 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fea6806da40 tx=0x7fea5802a1a0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.901+0000 7fea6ca0d640 1 -- 172.21.15.69:0/2397022395 shutdown_connections 2024-04-25T11:00:50.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.901+0000 7fea6ca0d640 1 --2- 172.21.15.69:0/2397022395 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea6806e8f0 0x7fea680d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.901+0000 7fea6ca0d640 1 --2- 172.21.15.69:0/2397022395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea680d4570 0x7fea680b4830 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.901+0000 7fea6ca0d640 1 -- 172.21.15.69:0/2397022395 >> 172.21.15.69:0/2397022395 conn(0x7fea680d3a50 msgr2=0x7fea680d3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:50.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.901+0000 7fea6ca0d640 1 -- 172.21.15.69:0/2397022395 shutdown_connections 2024-04-25T11:00:50.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.901+0000 7fea6ca0d640 1 -- 172.21.15.69:0/2397022395 wait complete. 2024-04-25T11:00:50.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.903+0000 7fea6ca0d640 1 Processor -- start 2024-04-25T11:00:50.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.903+0000 7fea6ca0d640 1 -- start start 2024-04-25T11:00:50.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.904+0000 7fea6ca0d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea6806e8f0 0x7fea68147d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:50.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.904+0000 7fea6ca0d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea680d09f0 0x7fea68161640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:50.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.904+0000 7fea6ca0d640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea680d4570 0x7fea681eaac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:50.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.904+0000 7fea6ca0d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fea680b34d0 con 0x7fea6806e8f0 2024-04-25T11:00:50.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.904+0000 7fea6ca0d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fea6806ed00 con 0x7fea680d09f0 2024-04-25T11:00:50.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.904+0000 7fea6ca0d640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fea680b3650 con 0x7fea680d4570 2024-04-25T11:00:50.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.904+0000 7fea6659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea6806e8f0 0x7fea68147d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:50.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.904+0000 7fea6659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea6806e8f0 0x7fea68147d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47948/0 (socket says 172.21.15.69:47948) 2024-04-25T11:00:50.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.904+0000 7fea6659c640 1 -- 172.21.15.69:0/1607617797 learned_addr learned my addr 172.21.15.69:0/1607617797 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:50.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.904+0000 7fea66d9d640 1 --2- 172.21.15.69:0/1607617797 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea680d4570 0x7fea681eaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:50.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.904+0000 7fea5dd9b640 1 --2- 172.21.15.69:0/1607617797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea680d09f0 0x7fea68161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:50.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.904+0000 7fea6659c640 1 -- 172.21.15.69:0/1607617797 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea680d4570 msgr2=0x7fea681eaac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.904+0000 7fea6659c640 1 --2- 172.21.15.69:0/1607617797 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea680d4570 0x7fea681eaac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.904+0000 7fea6659c640 1 -- 172.21.15.69:0/1607617797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea680d09f0 msgr2=0x7fea68161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:50.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.904+0000 7fea6659c640 1 --2- 172.21.15.69:0/1607617797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea680d09f0 0x7fea68161640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:50.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.904+0000 7fea6659c640 1 -- 172.21.15.69:0/1607617797 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea681e9a20 con 0x7fea6806e8f0 2024-04-25T11:00:50.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.905+0000 7fea6659c640 1 --2- 172.21.15.69:0/1607617797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea6806e8f0 0x7fea68147d30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fea58000c00 tx=0x7fea580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:50.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.905+0000 7fea5f7fe640 1 -- 172.21.15.69:0/1607617797 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea58041020 con 0x7fea6806e8f0 2024-04-25T11:00:50.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.905+0000 7fea5f7fe640 1 -- 172.21.15.69:0/1607617797 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea58003c50 con 0x7fea6806e8f0 2024-04-25T11:00:50.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.905+0000 7fea6ca0d640 1 -- 172.21.15.69:0/1607617797 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea681e9d00 con 0x7fea6806e8f0 2024-04-25T11:00:50.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.905+0000 7fea5f7fe640 1 -- 172.21.15.69:0/1607617797 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea58004030 con 0x7fea6806e8f0 2024-04-25T11:00:50.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.905+0000 7fea6ca0d640 1 -- 172.21.15.69:0/1607617797 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea681ea2c0 con 0x7fea6806e8f0 2024-04-25T11:00:50.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.906+0000 7fea6ca0d640 1 -- 172.21.15.69:0/1607617797 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea68050740 con 0x7fea6806e8f0 2024-04-25T11:00:50.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.906+0000 7fea5f7fe640 1 -- 172.21.15.69:0/1607617797 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fea58044020 con 0x7fea6806e8f0 2024-04-25T11:00:50.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.907+0000 7fea5f7fe640 1 --2- 172.21.15.69:0/1607617797 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fea480744b0 0x7fea480769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:50.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.907+0000 7fea5f7fe640 1 -- 172.21.15.69:0/1607617797 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7fea580b5be0 con 0x7fea6806e8f0 2024-04-25T11:00:50.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.907+0000 7fea5dd9b640 1 --2- 172.21.15.69:0/1607617797 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fea480744b0 0x7fea480769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:50.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.907+0000 7fea5dd9b640 1 --2- 172.21.15.69:0/1607617797 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fea480744b0 0x7fea480769c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fea50006fd0 tx=0x7fea50008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:50.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:50.911+0000 7fea5f7fe640 1 -- 172.21.15.69:0/1607617797 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fea5804b030 con 0x7fea6806e8f0 2024-04-25T11:00:51.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.085+0000 7fea6ca0d640 1 -- 172.21.15.69:0/1607617797 --> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fea680d1220 con 0x7fea480744b0 2024-04-25T11:00:51.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.086+0000 7fea5f7fe640 1 -- 172.21.15.69:0/1607617797 <== mgr.4107 v2:172.21.15.69:6832/1107421267 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fea680d1220 con 0x7fea480744b0 2024-04-25T11:00:51.086 INFO:teuthology.orchestra.run.smithi069.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T11:00:51.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.088+0000 7fea6ca0d640 1 -- 172.21.15.69:0/1607617797 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fea480744b0 msgr2=0x7fea480769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.088+0000 7fea6ca0d640 1 --2- 172.21.15.69:0/1607617797 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fea480744b0 0x7fea480769c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fea50006fd0 tx=0x7fea50008040 comp rx=0 tx=0).stop 2024-04-25T11:00:51.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.088+0000 7fea6ca0d640 1 -- 172.21.15.69:0/1607617797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea6806e8f0 msgr2=0x7fea68147d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.088+0000 7fea6ca0d640 1 --2- 172.21.15.69:0/1607617797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea6806e8f0 0x7fea68147d30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fea58000c00 tx=0x7fea580046c0 comp rx=0 tx=0).stop 2024-04-25T11:00:51.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.088+0000 7fea6ca0d640 1 -- 172.21.15.69:0/1607617797 shutdown_connections 2024-04-25T11:00:51.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.088+0000 7fea6ca0d640 1 --2- 172.21.15.69:0/1607617797 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fea480744b0 0x7fea480769c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:51.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.088+0000 7fea6ca0d640 1 --2- 172.21.15.69:0/1607617797 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea680d4570 0x7fea681eaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:51.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.088+0000 7fea6ca0d640 1 --2- 172.21.15.69:0/1607617797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea680d09f0 0x7fea68161640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:51.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.088+0000 7fea6ca0d640 1 --2- 172.21.15.69:0/1607617797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea6806e8f0 0x7fea68147d30 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:51.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.088+0000 7fea6ca0d640 1 -- 172.21.15.69:0/1607617797 >> 172.21.15.69:0/1607617797 conn(0x7fea680d3a50 msgr2=0x7fea680ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:51.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.088+0000 7fea6ca0d640 1 -- 172.21.15.69:0/1607617797 shutdown_connections 2024-04-25T11:00:51.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.088+0000 7fea6ca0d640 1 -- 172.21.15.69:0/1607617797 wait complete. 2024-04-25T11:00:51.099 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T11:00:51.100 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:00:51.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.211+0000 7fabfbe34640 1 Processor -- start 2024-04-25T11:00:51.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.211+0000 7fabfbe34640 1 -- start start 2024-04-25T11:00:51.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.213+0000 7fabfbe34640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fabf40b0c90 0x7fabf40704c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:51.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.213+0000 7fabfbe34640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fabf406dce0 0x7fabf40cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:51.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.213+0000 7fabfbe34640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabf406fa20 con 0x7fabf415adf0 2024-04-25T11:00:51.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.213+0000 7fabfbe34640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fabf4070fd0 con 0x7fabf406dce0 2024-04-25T11:00:51.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.213+0000 7fabfbe34640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fabf40712b0 con 0x7fabf40b0c90 2024-04-25T11:00:51.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.213+0000 7fabfa3d1640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fabf406dce0 0x7fabf40cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:51.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.213+0000 7fabfa3d1640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fabf406dce0 0x7fabf40cec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47960/0 (socket says 172.21.15.69:47960) 2024-04-25T11:00:51.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.213+0000 7fabf93cf640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fabf415adf0 0x7fabf406d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:54466/0 (socket says 172.21.15.69:54466) 2024-04-25T11:00:51.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.213+0000 7fabfa3d1640 1 -- 172.21.15.69:0/3962568693 learned_addr learned my addr 172.21.15.69:0/3962568693 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:51.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.213+0000 7fabf9bd0640 1 --2- 172.21.15.69:0/3962568693 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fabf40b0c90 0x7fabf40704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:51.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.213+0000 7fabfa3d1640 1 -- 172.21.15.69:0/3962568693 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fabf40b0c90 msgr2=0x7fabf40704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:51.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.213+0000 7fabfa3d1640 1 --2- 172.21.15.69:0/3962568693 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fabf40b0c90 0x7fabf40704c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:51.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.213+0000 7fabfa3d1640 1 -- 172.21.15.69:0/3962568693 >> v1:172.21.15.69:6789/0 conn(0x7fabf415adf0 legacy=0x7fabf406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.213+0000 7fabfa3d1640 1 -- 172.21.15.69:0/3962568693 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf406fcc0 con 0x7fabf406dce0 2024-04-25T11:00:51.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.213+0000 7fabf8bce640 1 -- 172.21.15.69:0/3962568693 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4115956461 0 0) 0x7fabf406fa20 con 0x7fabf415adf0 2024-04-25T11:00:51.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.214+0000 7fabfa3d1640 1 --2- 172.21.15.69:0/3962568693 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabf406dce0 0x7fabf40cec50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fabec009870 tx=0x7fabec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcbf57997925adab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:51.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.214+0000 7fabf8bce640 1 -- 172.21.15.69:0/3962568693 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabec037070 con 0x7fabf406dce0 2024-04-25T11:00:51.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.214+0000 7fabf8bce640 1 -- 172.21.15.69:0/3962568693 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabec032440 con 0x7fabf406dce0 2024-04-25T11:00:51.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.214+0000 7fabf8bce640 1 -- 172.21.15.69:0/3962568693 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabec032810 con 0x7fabf406dce0 2024-04-25T11:00:51.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.215+0000 7fabfbe34640 1 -- 172.21.15.69:0/3962568693 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabf406dce0 msgr2=0x7fabf40cec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.215+0000 7fabfbe34640 1 --2- 172.21.15.69:0/3962568693 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabf406dce0 0x7fabf40cec50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fabec009870 tx=0x7fabec02a270 comp rx=0 tx=0).stop 2024-04-25T11:00:51.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.215+0000 7fabfbe34640 1 -- 172.21.15.69:0/3962568693 shutdown_connections 2024-04-25T11:00:51.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.215+0000 7fabfbe34640 1 --2- 172.21.15.69:0/3962568693 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabf406dce0 0x7fabf40cec50 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:51.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.215+0000 7fabfbe34640 1 --2- 172.21.15.69:0/3962568693 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fabf40b0c90 0x7fabf40704c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:51.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.215+0000 7fabfbe34640 1 -- 172.21.15.69:0/3962568693 >> 172.21.15.69:0/3962568693 conn(0x7fabf4073090 msgr2=0x7fabf40734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:51.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.215+0000 7fabfbe34640 1 -- 172.21.15.69:0/3962568693 shutdown_connections 2024-04-25T11:00:51.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.215+0000 7fabfbe34640 1 -- 172.21.15.69:0/3962568693 wait complete. 2024-04-25T11:00:51.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.217+0000 7fabfbe34640 1 Processor -- start 2024-04-25T11:00:51.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.217+0000 7fabfbe34640 1 -- start start 2024-04-25T11:00:51.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.218+0000 7fabfbe34640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fabf40b0c90 0x7fabf41570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:51.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.218+0000 7fabfbe34640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabf415adf0 0x7fabf413b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:51.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.218+0000 7fabfbe34640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabf40b6140 0x7fabf4154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:51.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.218+0000 7fabfbe34640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fabf415d790 con 0x7fabf415adf0 2024-04-25T11:00:51.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.218+0000 7fabfbe34640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fabf4070c80 con 0x7fabf40b6140 2024-04-25T11:00:51.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.218+0000 7fabfbe34640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fabf415d910 con 0x7fabf40b0c90 2024-04-25T11:00:51.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.218+0000 7fabf93cf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabf415adf0 0x7fabf413b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:51.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.218+0000 7fabf93cf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabf415adf0 0x7fabf413b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47970/0 (socket says 172.21.15.69:47970) 2024-04-25T11:00:51.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.218+0000 7fabf93cf640 1 -- 172.21.15.69:0/1643083731 learned_addr learned my addr 172.21.15.69:0/1643083731 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:51.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.218+0000 7fabf93cf640 1 -- 172.21.15.69:0/1643083731 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fabf40b0c90 msgr2=0x7fabf41570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.218+0000 7fabf93cf640 1 --2- 172.21.15.69:0/1643083731 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fabf40b0c90 0x7fabf41570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:51.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.218+0000 7fabf93cf640 1 -- 172.21.15.69:0/1643083731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabf40b6140 msgr2=0x7fabf4154ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.218+0000 7fabf93cf640 1 --2- 172.21.15.69:0/1643083731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabf40b6140 0x7fabf4154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:51.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.218+0000 7fabf93cf640 1 -- 172.21.15.69:0/1643083731 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf413b780 con 0x7fabf415adf0 2024-04-25T11:00:51.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.218+0000 7fabf93cf640 1 --2- 172.21.15.69:0/1643083731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabf415adf0 0x7fabf413b240 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fabf406fbf0 tx=0x7fabe400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:51.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.218+0000 7fabea7fc640 1 -- 172.21.15.69:0/1643083731 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe40074e0 con 0x7fabf415adf0 2024-04-25T11:00:51.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.219+0000 7fabea7fc640 1 -- 172.21.15.69:0/1643083731 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabe4007680 con 0x7fabf415adf0 2024-04-25T11:00:51.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.219+0000 7fabfbe34640 1 -- 172.21.15.69:0/1643083731 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabf41579b0 con 0x7fabf415adf0 2024-04-25T11:00:51.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.219+0000 7fabea7fc640 1 -- 172.21.15.69:0/1643083731 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe4004a20 con 0x7fabf415adf0 2024-04-25T11:00:51.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.219+0000 7fabfbe34640 1 -- 172.21.15.69:0/1643083731 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabf415fae0 con 0x7fabf415adf0 2024-04-25T11:00:51.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.219+0000 7fabfbe34640 1 -- 172.21.15.69:0/1643083731 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fabf4050740 con 0x7fabf415adf0 2024-04-25T11:00:51.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.220+0000 7fabea7fc640 1 -- 172.21.15.69:0/1643083731 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fabe401a020 con 0x7fabf415adf0 2024-04-25T11:00:51.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.221+0000 7fabea7fc640 1 --2- 172.21.15.69:0/1643083731 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fabd80744e0 0x7fabd80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:51.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.221+0000 7fabea7fc640 1 -- 172.21.15.69:0/1643083731 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7fabe40951c0 con 0x7fabf415adf0 2024-04-25T11:00:51.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.221+0000 7fabf9bd0640 1 --2- 172.21.15.69:0/1643083731 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fabd80744e0 0x7fabd80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:51.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.221+0000 7fabea7fc640 1 --2- 172.21.15.69:0/1643083731 >> [v2:172.21.15.146:6800/2508929872,v1:172.21.15.146:6801/2508929872] conn(0x7fabd807a490 0x7fabd807c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:51.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.221+0000 7fabea7fc640 1 -- 172.21.15.69:0/1643083731 --> [v2:172.21.15.146:6800/2508929872,v1:172.21.15.146:6801/2508929872] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fabd807cfb0 con 0x7fabd807a490 2024-04-25T11:00:51.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.221+0000 7fabea7fc640 1 -- 172.21.15.69:0/1643083731 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fabe4095650 con 0x7fabf415adf0 2024-04-25T11:00:51.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.221+0000 7fabf9bd0640 1 --2- 172.21.15.69:0/1643083731 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fabd80744e0 0x7fabd80769f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fabf4073040 tx=0x7fabf0009830 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:51.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.221+0000 7fabfa3d1640 1 --2- 172.21.15.69:0/1643083731 >> [v2:172.21.15.146:6800/2508929872,v1:172.21.15.146:6801/2508929872] conn(0x7fabd807a490 0x7fabd807c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:51.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.222+0000 7fabfa3d1640 1 --2- 172.21.15.69:0/1643083731 >> [v2:172.21.15.146:6800/2508929872,v1:172.21.15.146:6801/2508929872] conn(0x7fabd807a490 0x7fabd807c8e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:51.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.223+0000 7fabea7fc640 1 -- 172.21.15.69:0/1643083731 <== osd.4 v2:172.21.15.146:6800/2508929872 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fabd807cfb0 con 0x7fabd807a490 2024-04-25T11:00:51.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.236+0000 7fabfbe34640 1 -- 172.21.15.69:0/1643083731 --> [v2:172.21.15.146:6800/2508929872,v1:172.21.15.146:6801/2508929872] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fabf4157760 con 0x7fabd807a490 2024-04-25T11:00:51.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.236+0000 7fabea7fc640 1 -- 172.21.15.69:0/1643083731 <== osd.4 v2:172.21.15.146:6800/2508929872 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fabf4157760 con 0x7fabd807a490 2024-04-25T11:00:51.237 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T11:00:51.237 INFO:teuthology.orchestra.run.smithi069.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:00:51.237 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T11:00:51.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.236+0000 7fabfbe34640 1 -- 172.21.15.69:0/1643083731 >> [v2:172.21.15.146:6800/2508929872,v1:172.21.15.146:6801/2508929872] conn(0x7fabd807a490 msgr2=0x7fabd807c8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.236+0000 7fabfbe34640 1 --2- 172.21.15.69:0/1643083731 >> [v2:172.21.15.146:6800/2508929872,v1:172.21.15.146:6801/2508929872] conn(0x7fabd807a490 0x7fabd807c8e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:51.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.237+0000 7fabfbe34640 1 -- 172.21.15.69:0/1643083731 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fabd80744e0 msgr2=0x7fabd80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.237+0000 7fabfbe34640 1 --2- 172.21.15.69:0/1643083731 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fabd80744e0 0x7fabd80769f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fabf4073040 tx=0x7fabf0009830 comp rx=0 tx=0).stop 2024-04-25T11:00:51.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.237+0000 7fabfbe34640 1 -- 172.21.15.69:0/1643083731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabf415adf0 msgr2=0x7fabf413b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.237+0000 7fabfbe34640 1 --2- 172.21.15.69:0/1643083731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabf415adf0 0x7fabf413b240 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fabf406fbf0 tx=0x7fabe400c820 comp rx=0 tx=0).stop 2024-04-25T11:00:51.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.237+0000 7fabfa3d1640 1 -- 172.21.15.69:0/1643083731 reap_dead start 2024-04-25T11:00:51.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.237+0000 7fabfbe34640 1 -- 172.21.15.69:0/1643083731 shutdown_connections 2024-04-25T11:00:51.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.237+0000 7fabfbe34640 1 -- 172.21.15.69:0/1643083731 >> 172.21.15.69:0/1643083731 conn(0x7fabf4073090 msgr2=0x7fabf415c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:51.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.237+0000 7fabfbe34640 1 -- 172.21.15.69:0/1643083731 shutdown_connections 2024-04-25T11:00:51.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.237+0000 7fabfbe34640 1 -- 172.21.15.69:0/1643083731 wait complete. 2024-04-25T11:00:51.248 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T11:00:51.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.356+0000 7f9f81b7a640 1 Processor -- start 2024-04-25T11:00:51.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.356+0000 7f9f81b7a640 1 -- start start 2024-04-25T11:00:51.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.357+0000 7f9f81b7a640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f9f7c0d1bc0 0x7f9f7c0b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:51.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.357+0000 7f9f81b7a640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f7c0b0770 con 0x7f9f7c0d1620 2024-04-25T11:00:51.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.357+0000 7f9f81b7a640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f7c0b0a50 con 0x7f9f7c0ce800 2024-04-25T11:00:51.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.357+0000 7f9f81b7a640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7f9f7c0b10a0 con 0x7f9f7c0d1bc0 2024-04-25T11:00:51.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.357+0000 7f9f7bfff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7f9f7c0d1bc0 0x7f9f7c0b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:51.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.357+0000 7f9f7affd640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f9f7c0d1620 0x7f9f7c163130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:54470/0 (socket says 172.21.15.69:54470) 2024-04-25T11:00:51.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.358+0000 7f9f7affd640 1 -- 172.21.15.69:0/1228341689 learned_addr learned my addr 172.21.15.69:0/1228341689 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:51.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.358+0000 7f9f7bfff640 1 -- 172.21.15.69:0/1228341689 >> v1:172.21.15.146:6789/0 conn(0x7f9f7c0ce800 legacy=0x7f9f7c162a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:51.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.358+0000 7f9f7bfff640 1 -- 172.21.15.69:0/1228341689 >> v1:172.21.15.69:6789/0 conn(0x7f9f7c0d1620 legacy=0x7f9f7c163130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.358+0000 7f9f7bfff640 1 -- 172.21.15.69:0/1228341689 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f7c0b0d30 con 0x7f9f7c0d1bc0 2024-04-25T11:00:51.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.358+0000 7f9f7a7fc640 1 -- 172.21.15.69:0/1228341689 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 326732633 0 0) 0x7f9f7c0b0770 con 0x7f9f7c0d1620 2024-04-25T11:00:51.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.358+0000 7f9f7bfff640 1 --2- 172.21.15.69:0/1228341689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f7c0d1bc0 0x7f9f7c0b3c40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c0b1220 tx=0x7f9f7402a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d458600c9ad0808 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:51.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.359+0000 7f9f7a7fc640 1 -- 172.21.15.69:0/1228341689 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f74007740 con 0x7f9f7c0d1bc0 2024-04-25T11:00:51.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.359+0000 7f9f7a7fc640 1 -- 172.21.15.69:0/1228341689 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f740078e0 con 0x7f9f7c0d1bc0 2024-04-25T11:00:51.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.359+0000 7f9f7a7fc640 1 -- 172.21.15.69:0/1228341689 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f74007cb0 con 0x7f9f7c0d1bc0 2024-04-25T11:00:51.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.359+0000 7f9f81b7a640 1 -- 172.21.15.69:0/1228341689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f7c0d1bc0 msgr2=0x7f9f7c0b3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.359+0000 7f9f81b7a640 1 --2- 172.21.15.69:0/1228341689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f7c0d1bc0 0x7f9f7c0b3c40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c0b1220 tx=0x7f9f7402a570 comp rx=0 tx=0).stop 2024-04-25T11:00:51.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.359+0000 7f9f81b7a640 1 -- 172.21.15.69:0/1228341689 shutdown_connections 2024-04-25T11:00:51.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.359+0000 7f9f81b7a640 1 --2- 172.21.15.69:0/1228341689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f7c0d1bc0 0x7f9f7c0b3c40 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:51.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.359+0000 7f9f81b7a640 1 -- 172.21.15.69:0/1228341689 >> 172.21.15.69:0/1228341689 conn(0x7f9f7c0d3a50 msgr2=0x7f9f7c0d3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:51.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.359+0000 7f9f81b7a640 1 -- 172.21.15.69:0/1228341689 shutdown_connections 2024-04-25T11:00:51.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.359+0000 7f9f81b7a640 1 -- 172.21.15.69:0/1228341689 wait complete. 2024-04-25T11:00:51.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.361+0000 7f9f81b7a640 1 Processor -- start 2024-04-25T11:00:51.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.361+0000 7f9f81b7a640 1 -- start start 2024-04-25T11:00:51.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.363+0000 7f9f81b7a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f7c0ce800 0x7f9f7c1f5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:51.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.363+0000 7f9f81b7a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f7c0d1620 0x7f9f7c1e9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:51.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.363+0000 7f9f81b7a640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f7c0d1bc0 0x7f9f7c1f4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:51.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.363+0000 7f9f81b7a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7c0b4180 con 0x7f9f7c0d1620 2024-04-25T11:00:51.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.363+0000 7f9f81b7a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7c0d1fd0 con 0x7f9f7c0ce800 2024-04-25T11:00:51.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.363+0000 7f9f81b7a640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f9f7c0b4300 con 0x7f9f7c0d1bc0 2024-04-25T11:00:51.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.363+0000 7f9f7affd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f7c0d1620 0x7f9f7c1e9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:51.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.363+0000 7f9f7affd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f7c0d1620 0x7f9f7c1e9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48000/0 (socket says 172.21.15.69:48000) 2024-04-25T11:00:51.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.363+0000 7f9f7affd640 1 -- 172.21.15.69:0/1990354383 learned_addr learned my addr 172.21.15.69:0/1990354383 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:51.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.363+0000 7f9f7b7fe640 1 --2- 172.21.15.69:0/1990354383 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f7c0ce800 0x7f9f7c1f5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:51.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.363+0000 7f9f7bfff640 1 --2- 172.21.15.69:0/1990354383 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f7c0d1bc0 0x7f9f7c1f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:51.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.363+0000 7f9f7affd640 1 -- 172.21.15.69:0/1990354383 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f7c0d1bc0 msgr2=0x7f9f7c1f4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.363+0000 7f9f7affd640 1 --2- 172.21.15.69:0/1990354383 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f7c0d1bc0 0x7f9f7c1f4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:51.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.363+0000 7f9f7affd640 1 -- 172.21.15.69:0/1990354383 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f7c0ce800 msgr2=0x7f9f7c1f5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.363+0000 7f9f7affd640 1 --2- 172.21.15.69:0/1990354383 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f7c0ce800 0x7f9f7c1f5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:51.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.363+0000 7f9f7affd640 1 -- 172.21.15.69:0/1990354383 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f7c1e8de0 con 0x7f9f7c0d1620 2024-04-25T11:00:51.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.363+0000 7f9f7affd640 1 --2- 172.21.15.69:0/1990354383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f7c0d1620 0x7f9f7c1e9e80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c00c3e0 tx=0x7f9f6c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:51.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.363+0000 7f9f73fff640 1 -- 172.21.15.69:0/1990354383 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c0074e0 con 0x7f9f7c0d1620 2024-04-25T11:00:51.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.363+0000 7f9f73fff640 1 -- 172.21.15.69:0/1990354383 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f6c0047e0 con 0x7f9f7c0d1620 2024-04-25T11:00:51.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.363+0000 7f9f81b7a640 1 -- 172.21.15.69:0/1990354383 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f7c1e90c0 con 0x7f9f7c0d1620 2024-04-25T11:00:51.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.363+0000 7f9f73fff640 1 -- 172.21.15.69:0/1990354383 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c004bd0 con 0x7f9f7c0d1620 2024-04-25T11:00:51.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.363+0000 7f9f81b7a640 1 -- 172.21.15.69:0/1990354383 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f7c1e9680 con 0x7f9f7c0d1620 2024-04-25T11:00:51.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.364+0000 7f9f81b7a640 1 -- 172.21.15.69:0/1990354383 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f7c1e8b50 con 0x7f9f7c0d1620 2024-04-25T11:00:51.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.365+0000 7f9f73fff640 1 -- 172.21.15.69:0/1990354383 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9f6c011070 con 0x7f9f7c0d1620 2024-04-25T11:00:51.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.365+0000 7f9f73fff640 1 --2- 172.21.15.69:0/1990354383 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f9f5c0744e0 0x7f9f5c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:51.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.366+0000 7f9f73fff640 1 -- 172.21.15.69:0/1990354383 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7f9f6c094b40 con 0x7f9f7c0d1620 2024-04-25T11:00:51.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.366+0000 7f9f7b7fe640 1 --2- 172.21.15.69:0/1990354383 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f9f5c0744e0 0x7f9f5c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:51.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.366+0000 7f9f7b7fe640 1 --2- 172.21.15.69:0/1990354383 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f9f5c0744e0 0x7f9f5c0769f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9f6400b7f0 tx=0x7f9f64008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:51.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.370+0000 7f9f73fff640 1 -- 172.21.15.69:0/1990354383 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9f6c05f570 con 0x7f9f7c0d1620 2024-04-25T11:00:51.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.545+0000 7f9f81b7a640 1 -- 172.21.15.69:0/1990354383 --> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f9f7c050740 con 0x7f9f5c0744e0 2024-04-25T11:00:51.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.545+0000 7f9f73fff640 1 -- 172.21.15.69:0/1990354383 <== mgr.4107 v2:172.21.15.69:6832/1107421267 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9f7c050740 con 0x7f9f5c0744e0 2024-04-25T11:00:51.546 INFO:teuthology.orchestra.run.smithi069.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T11:00:51.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.547+0000 7f9f81b7a640 1 -- 172.21.15.69:0/1990354383 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f9f5c0744e0 msgr2=0x7f9f5c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.547+0000 7f9f81b7a640 1 --2- 172.21.15.69:0/1990354383 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f9f5c0744e0 0x7f9f5c0769f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9f6400b7f0 tx=0x7f9f64008040 comp rx=0 tx=0).stop 2024-04-25T11:00:51.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.547+0000 7f9f81b7a640 1 -- 172.21.15.69:0/1990354383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f7c0d1620 msgr2=0x7f9f7c1e9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.547+0000 7f9f81b7a640 1 --2- 172.21.15.69:0/1990354383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f7c0d1620 0x7f9f7c1e9e80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c00c3e0 tx=0x7f9f6c00c8a0 comp rx=0 tx=0).stop 2024-04-25T11:00:51.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.547+0000 7f9f81b7a640 1 -- 172.21.15.69:0/1990354383 shutdown_connections 2024-04-25T11:00:51.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.547+0000 7f9f81b7a640 1 --2- 172.21.15.69:0/1990354383 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f9f5c0744e0 0x7f9f5c0769f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:51.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.547+0000 7f9f81b7a640 1 --2- 172.21.15.69:0/1990354383 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f7c0d1bc0 0x7f9f7c1f4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:51.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.547+0000 7f9f81b7a640 1 --2- 172.21.15.69:0/1990354383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f7c0d1620 0x7f9f7c1e9e80 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:51.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.547+0000 7f9f81b7a640 1 --2- 172.21.15.69:0/1990354383 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f7c0ce800 0x7f9f7c1f5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:51.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.547+0000 7f9f81b7a640 1 -- 172.21.15.69:0/1990354383 >> 172.21.15.69:0/1990354383 conn(0x7f9f7c0d3a50 msgr2=0x7f9f7c0cff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:51.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.548+0000 7f9f81b7a640 1 -- 172.21.15.69:0/1990354383 shutdown_connections 2024-04-25T11:00:51.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.548+0000 7f9f81b7a640 1 -- 172.21.15.69:0/1990354383 wait complete. 2024-04-25T11:00:51.559 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T11:00:51.559 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:00:51.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.670+0000 7fc57d642640 1 Processor -- start 2024-04-25T11:00:51.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.670+0000 7fc57d642640 1 -- start start 2024-04-25T11:00:51.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.671+0000 7fc57d642640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc57806e0b0 0x7fc5780b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:51.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.671+0000 7fc57d642640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5781640b0 0x7fc5781644c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:51.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.671+0000 7fc57d642640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc57806e650 con 0x7fc578164a00 2024-04-25T11:00:51.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.671+0000 7fc57d642640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fc57806e930 con 0x7fc57806e0b0 2024-04-25T11:00:51.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.671+0000 7fc57d642640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc57806ec10 con 0x7fc5781640b0 2024-04-25T11:00:51.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.671+0000 7fc56e7fc640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5781640b0 0x7fc5781644c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:51.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.671+0000 7fc576ffd640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc57806e0b0 0x7fc5780b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:51.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.671+0000 7fc56e7fc640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5781640b0 0x7fc5781644c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.69:37794/0 (socket says 172.21.15.69:37794) 2024-04-25T11:00:51.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.671+0000 7fc5777fe640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fc578164a00 0x7fc5780d2020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.69:53990/0 (socket says 172.21.15.69:53990) 2024-04-25T11:00:51.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.672+0000 7fc56e7fc640 1 -- 172.21.15.69:0/2522888518 learned_addr learned my addr 172.21.15.69:0/2522888518 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:51.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.672+0000 7fc56e7fc640 1 -- 172.21.15.69:0/2522888518 >> v2:172.21.15.146:3300/0 conn(0x7fc57806e0b0 msgr2=0x7fc5780b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:51.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.672+0000 7fc56e7fc640 1 --2- 172.21.15.69:0/2522888518 >> v2:172.21.15.146:3300/0 conn(0x7fc57806e0b0 0x7fc5780b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:51.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.672+0000 7fc56e7fc640 1 -- 172.21.15.69:0/2522888518 >> v1:172.21.15.146:6789/0 conn(0x7fc578164a00 legacy=0x7fc5780d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.672+0000 7fc56e7fc640 1 -- 172.21.15.69:0/2522888518 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5780b07f0 con 0x7fc5781640b0 2024-04-25T11:00:51.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.672+0000 7fc576ffd640 1 --2- 172.21.15.69:0/2522888518 >> v2:172.21.15.146:3300/0 conn(0x7fc57806e0b0 0x7fc5780b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:00:51.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.673+0000 7fc56e7fc640 1 --2- 172.21.15.69:0/2522888518 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5781640b0 0x7fc5781644c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc560009870 tx=0x7fc56002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47ec460b3225308d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:51.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.673+0000 7fc5767fc640 1 -- 172.21.15.69:0/2522888518 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc560037070 con 0x7fc5781640b0 2024-04-25T11:00:51.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.673+0000 7fc5767fc640 1 -- 172.21.15.69:0/2522888518 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc560032440 con 0x7fc5781640b0 2024-04-25T11:00:51.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.673+0000 7fc5767fc640 1 -- 172.21.15.69:0/2522888518 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc560032810 con 0x7fc5781640b0 2024-04-25T11:00:51.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.674+0000 7fc57d642640 1 -- 172.21.15.69:0/2522888518 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5781640b0 msgr2=0x7fc5781644c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.674+0000 7fc57d642640 1 --2- 172.21.15.69:0/2522888518 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5781640b0 0x7fc5781644c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc560009870 tx=0x7fc56002a270 comp rx=0 tx=0).stop 2024-04-25T11:00:51.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.674+0000 7fc57d642640 1 -- 172.21.15.69:0/2522888518 shutdown_connections 2024-04-25T11:00:51.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.674+0000 7fc57d642640 1 --2- 172.21.15.69:0/2522888518 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5781640b0 0x7fc5781644c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:51.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.674+0000 7fc57d642640 1 --2- 172.21.15.69:0/2522888518 >> v2:172.21.15.146:3300/0 conn(0x7fc57806e0b0 0x7fc5780b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:51.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.674+0000 7fc57d642640 1 -- 172.21.15.69:0/2522888518 >> 172.21.15.69:0/2522888518 conn(0x7fc5780d3a60 msgr2=0x7fc5780d3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:51.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.674+0000 7fc57d642640 1 -- 172.21.15.69:0/2522888518 shutdown_connections 2024-04-25T11:00:51.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.674+0000 7fc57d642640 1 -- 172.21.15.69:0/2522888518 wait complete. 2024-04-25T11:00:51.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.675+0000 7fc57d642640 1 Processor -- start 2024-04-25T11:00:51.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.676+0000 7fc57d642640 1 -- start start 2024-04-25T11:00:51.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.677+0000 7fc57d642640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc57806e0b0 0x7fc578159da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:51.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.677+0000 7fc57d642640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5781640b0 0x7fc5780cce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:51.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.677+0000 7fc57d642640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc578164a00 0x7fc57815e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:51.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.677+0000 7fc57d642640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc5780d33d0 con 0x7fc5781640b0 2024-04-25T11:00:51.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.677+0000 7fc57d642640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc5780d3250 con 0x7fc57806e0b0 2024-04-25T11:00:51.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.677+0000 7fc57d642640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc5780b1d20 con 0x7fc578164a00 2024-04-25T11:00:51.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.677+0000 7fc56e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5781640b0 0x7fc5780cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:51.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.677+0000 7fc56e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5781640b0 0x7fc5780cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48004/0 (socket says 172.21.15.69:48004) 2024-04-25T11:00:51.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.677+0000 7fc56e7fc640 1 -- 172.21.15.69:0/2611990011 learned_addr learned my addr 172.21.15.69:0/2611990011 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:51.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.677+0000 7fc576ffd640 1 --2- 172.21.15.69:0/2611990011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc57806e0b0 0x7fc578159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:51.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.677+0000 7fc5777fe640 1 --2- 172.21.15.69:0/2611990011 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc578164a00 0x7fc57815e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:51.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.677+0000 7fc56e7fc640 1 -- 172.21.15.69:0/2611990011 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc578164a00 msgr2=0x7fc57815e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.677+0000 7fc56e7fc640 1 --2- 172.21.15.69:0/2611990011 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc578164a00 0x7fc57815e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:51.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.677+0000 7fc56e7fc640 1 -- 172.21.15.69:0/2611990011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc57806e0b0 msgr2=0x7fc578159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.677+0000 7fc56e7fc640 1 --2- 172.21.15.69:0/2611990011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc57806e0b0 0x7fc578159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:51.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.677+0000 7fc56e7fc640 1 -- 172.21.15.69:0/2611990011 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5780cd3b0 con 0x7fc5781640b0 2024-04-25T11:00:51.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.678+0000 7fc56e7fc640 1 --2- 172.21.15.69:0/2611990011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5781640b0 0x7fc5780cce70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc560000c00 tx=0x7fc5600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:51.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.678+0000 7fc56ffff640 1 -- 172.21.15.69:0/2611990011 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc560037040 con 0x7fc5781640b0 2024-04-25T11:00:51.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.678+0000 7fc56ffff640 1 -- 172.21.15.69:0/2611990011 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc560042070 con 0x7fc5781640b0 2024-04-25T11:00:51.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.678+0000 7fc57d642640 1 -- 172.21.15.69:0/2611990011 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5780cbdd0 con 0x7fc5781640b0 2024-04-25T11:00:51.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.678+0000 7fc56ffff640 1 -- 172.21.15.69:0/2611990011 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc560030880 con 0x7fc5781640b0 2024-04-25T11:00:51.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.678+0000 7fc57d642640 1 -- 172.21.15.69:0/2611990011 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5780cc390 con 0x7fc5781640b0 2024-04-25T11:00:51.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.679+0000 7fc57d642640 1 -- 172.21.15.69:0/2611990011 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc5780cca50 con 0x7fc5781640b0 2024-04-25T11:00:51.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.679+0000 7fc56ffff640 1 -- 172.21.15.69:0/2611990011 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc560030a20 con 0x7fc5781640b0 2024-04-25T11:00:51.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.680+0000 7fc56ffff640 1 --2- 172.21.15.69:0/2611990011 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc5580744b0 0x7fc5580769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:51.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.680+0000 7fc576ffd640 1 --2- 172.21.15.69:0/2611990011 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc5580744b0 0x7fc5580769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:51.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.680+0000 7fc56ffff640 1 -- 172.21.15.69:0/2611990011 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7fc5600b6120 con 0x7fc5781640b0 2024-04-25T11:00:51.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.680+0000 7fc56ffff640 1 --2- 172.21.15.69:0/2611990011 >> [v2:172.21.15.146:6809/1310911138,v1:172.21.15.146:6811/1310911138] conn(0x7fc55807a460 0x7fc55807c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:51.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.680+0000 7fc56ffff640 1 -- 172.21.15.69:0/2611990011 --> [v2:172.21.15.146:6809/1310911138,v1:172.21.15.146:6811/1310911138] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc560051e50 con 0x7fc55807a460 2024-04-25T11:00:51.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.680+0000 7fc56ffff640 1 -- 172.21.15.69:0/2611990011 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc5600b65b0 con 0x7fc5781640b0 2024-04-25T11:00:51.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.680+0000 7fc576ffd640 1 --2- 172.21.15.69:0/2611990011 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc5580744b0 0x7fc5580769c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc568009560 tx=0x7fc568009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:51.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.680+0000 7fc5777fe640 1 --2- 172.21.15.69:0/2611990011 >> [v2:172.21.15.146:6809/1310911138,v1:172.21.15.146:6811/1310911138] conn(0x7fc55807a460 0x7fc55807c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:51.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.681+0000 7fc5777fe640 1 --2- 172.21.15.69:0/2611990011 >> [v2:172.21.15.146:6809/1310911138,v1:172.21.15.146:6811/1310911138] conn(0x7fc55807a460 0x7fc55807c8b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:51.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.682+0000 7fc56ffff640 1 -- 172.21.15.69:0/2611990011 <== osd.5 v2:172.21.15.146:6809/1310911138 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc560051e50 con 0x7fc55807a460 2024-04-25T11:00:51.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.694+0000 7fc57d642640 1 -- 172.21.15.69:0/2611990011 --> [v2:172.21.15.146:6809/1310911138,v1:172.21.15.146:6811/1310911138] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc578050740 con 0x7fc55807a460 2024-04-25T11:00:51.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.695+0000 7fc56ffff640 1 -- 172.21.15.69:0/2611990011 <== osd.5 v2:172.21.15.146:6809/1310911138 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc578050740 con 0x7fc55807a460 2024-04-25T11:00:51.695 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T11:00:51.696 INFO:teuthology.orchestra.run.smithi069.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:00:51.696 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T11:00:51.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.695+0000 7fc57d642640 1 -- 172.21.15.69:0/2611990011 >> [v2:172.21.15.146:6809/1310911138,v1:172.21.15.146:6811/1310911138] conn(0x7fc55807a460 msgr2=0x7fc55807c8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.695+0000 7fc57d642640 1 --2- 172.21.15.69:0/2611990011 >> [v2:172.21.15.146:6809/1310911138,v1:172.21.15.146:6811/1310911138] conn(0x7fc55807a460 0x7fc55807c8b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:51.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.695+0000 7fc57d642640 1 -- 172.21.15.69:0/2611990011 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc5580744b0 msgr2=0x7fc5580769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.695+0000 7fc57d642640 1 --2- 172.21.15.69:0/2611990011 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc5580744b0 0x7fc5580769c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc568009560 tx=0x7fc568009190 comp rx=0 tx=0).stop 2024-04-25T11:00:51.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.695+0000 7fc57d642640 1 -- 172.21.15.69:0/2611990011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5781640b0 msgr2=0x7fc5780cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.695+0000 7fc57d642640 1 --2- 172.21.15.69:0/2611990011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5781640b0 0x7fc5780cce70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc560000c00 tx=0x7fc5600046c0 comp rx=0 tx=0).stop 2024-04-25T11:00:51.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.695+0000 7fc5777fe640 1 -- 172.21.15.69:0/2611990011 reap_dead start 2024-04-25T11:00:51.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.695+0000 7fc57d642640 1 -- 172.21.15.69:0/2611990011 shutdown_connections 2024-04-25T11:00:51.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.695+0000 7fc57d642640 1 -- 172.21.15.69:0/2611990011 >> 172.21.15.69:0/2611990011 conn(0x7fc5780d3a60 msgr2=0x7fc57815c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:51.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.696+0000 7fc57d642640 1 -- 172.21.15.69:0/2611990011 shutdown_connections 2024-04-25T11:00:51.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.696+0000 7fc57d642640 1 -- 172.21.15.69:0/2611990011 wait complete. 2024-04-25T11:00:51.706 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T11:00:51.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.814+0000 7f4ff9d9e640 1 Processor -- start 2024-04-25T11:00:51.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.814+0000 7f4ff9d9e640 1 -- start start 2024-04-25T11:00:51.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.816+0000 7f4ff9d9e640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4ff40d2410 0x7f4ff416b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:51.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.816+0000 7f4ff9d9e640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ff40b4390 con 0x7f4ff40b3f80 2024-04-25T11:00:51.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.816+0000 7f4ff9d9e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ff40cfb30 con 0x7f4ff4160870 2024-04-25T11:00:51.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.816+0000 7f4ff9d9e640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f4ff40b3910 con 0x7f4ff40d2410 2024-04-25T11:00:51.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.816+0000 7f4ff2ffd640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f4ff40b3f80 0x7f4ff40d1d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:54484/0 (socket says 172.21.15.69:54484) 2024-04-25T11:00:51.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.816+0000 7f4ff2ffd640 1 -- 172.21.15.69:0/4240614831 learned_addr learned my addr 172.21.15.69:0/4240614831 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:51.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.816+0000 7f4ff3fff640 1 --2- 172.21.15.69:0/4240614831 >> v2:172.21.15.146:3300/0 conn(0x7f4ff40d2410 0x7f4ff416b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:51.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.816+0000 7f4ff27fc640 1 -- 172.21.15.69:0/4240614831 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4061923754 0 0) 0x7f4fe8001110 con 0x7f4ff40b3f80 2024-04-25T11:00:51.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.816+0000 7f4ff27fc640 1 -- 172.21.15.69:0/4240614831 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4fdc003520 con 0x7f4ff40b3f80 2024-04-25T11:00:51.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.816+0000 7f4ff27fc640 1 -- 172.21.15.69:0/4240614831 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 54140188 0 0) 0x7f4fdc003520 con 0x7f4ff40b3f80 2024-04-25T11:00:51.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.816+0000 7f4ff27fc640 1 -- 172.21.15.69:0/4240614831 >> v2:172.21.15.146:3300/0 conn(0x7f4ff40d2410 msgr2=0x7f4ff416b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:51.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.816+0000 7f4ff27fc640 1 --2- 172.21.15.69:0/4240614831 >> v2:172.21.15.146:3300/0 conn(0x7f4ff40d2410 0x7f4ff416b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:51.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.816+0000 7f4ff27fc640 1 -- 172.21.15.69:0/4240614831 >> v1:172.21.15.146:6789/0 conn(0x7f4ff4160870 legacy=0x7f4ff40cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.816+0000 7f4ff27fc640 1 -- 172.21.15.69:0/4240614831 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff40cfe10 con 0x7f4ff40b3f80 2024-04-25T11:00:51.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.816+0000 7f4ff27fc640 1 -- 172.21.15.69:0/4240614831 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7f4fe8001840 con 0x7f4ff40b3f80 2024-04-25T11:00:51.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.817+0000 7f4ff27fc640 1 -- 172.21.15.69:0/4240614831 >> v1:172.21.15.69:6789/0 conn(0x7f4ff40b3f80 legacy=0x7f4ff40d1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.817+0000 7f4ff27fc640 1 --2- 172.21.15.69:0/4240614831 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4fdc0039b0 0x7f4fdc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:51.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.817+0000 7f4ff27fc640 1 --2- 172.21.15.69:0/4240614831 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4fdc004800 0x7f4fdc004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:51.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.817+0000 7f4ff27fc640 1 --2- 172.21.15.69:0/4240614831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fdc0051c0 0x7f4fdc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:51.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.817+0000 7f4ff27fc640 1 -- 172.21.15.69:0/4240614831 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4fdc00c130 con 0x7f4fdc0051c0 2024-04-25T11:00:51.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.817+0000 7f4ff27fc640 1 -- 172.21.15.69:0/4240614831 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4fdc00c2e0 con 0x7f4fdc0039b0 2024-04-25T11:00:51.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.817+0000 7f4ff27fc640 1 -- 172.21.15.69:0/4240614831 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4fdc00c4e0 con 0x7f4fdc004800 2024-04-25T11:00:51.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.817+0000 7f4ff3fff640 1 --2- 172.21.15.69:0/4240614831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fdc0051c0 0x7f4fdc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:51.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.817+0000 7f4ff37fe640 1 --2- 172.21.15.69:0/4240614831 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4fdc0039b0 0x7f4fdc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:51.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.817+0000 7f4ff2ffd640 1 --2- 172.21.15.69:0/4240614831 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4fdc004800 0x7f4fdc004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:51.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.817+0000 7f4ff3fff640 1 -- 172.21.15.69:0/4240614831 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4fdc004800 msgr2=0x7f4fdc004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:51.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.817+0000 7f4ff3fff640 1 --2- 172.21.15.69:0/4240614831 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4fdc004800 0x7f4fdc004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:51.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.817+0000 7f4ff3fff640 1 -- 172.21.15.69:0/4240614831 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4fdc0039b0 msgr2=0x7f4fdc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:51.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.817+0000 7f4ff3fff640 1 --2- 172.21.15.69:0/4240614831 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4fdc0039b0 0x7f4fdc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:51.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.817+0000 7f4ff3fff640 1 -- 172.21.15.69:0/4240614831 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fdc003520 con 0x7f4fdc0051c0 2024-04-25T11:00:51.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.818+0000 7f4ff3fff640 1 -- 172.21.15.69:0/4240614831 reap_dead start 2024-04-25T11:00:51.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.818+0000 7f4ff3fff640 1 --2- 172.21.15.69:0/4240614831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fdc0051c0 0x7f4fdc00b9c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4fec009fc0 tx=0x7f4fdc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34565e03d313f224 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:51.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.818+0000 7f4ff27fc640 1 -- 172.21.15.69:0/4240614831 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec009c10 con 0x7f4fdc0051c0 2024-04-25T11:00:51.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.818+0000 7f4ff27fc640 1 -- 172.21.15.69:0/4240614831 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fec003bd0 con 0x7f4fdc0051c0 2024-04-25T11:00:51.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.818+0000 7f4ff27fc640 1 -- 172.21.15.69:0/4240614831 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec004020 con 0x7f4fdc0051c0 2024-04-25T11:00:51.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.818+0000 7f4ff9d9e640 1 -- 172.21.15.69:0/4240614831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fdc0051c0 msgr2=0x7f4fdc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.818+0000 7f4ff9d9e640 1 --2- 172.21.15.69:0/4240614831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fdc0051c0 0x7f4fdc00b9c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4fec009fc0 tx=0x7f4fdc005830 comp rx=0 tx=0).stop 2024-04-25T11:00:51.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.819+0000 7f4ff9d9e640 1 -- 172.21.15.69:0/4240614831 shutdown_connections 2024-04-25T11:00:51.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.819+0000 7f4ff9d9e640 1 --2- 172.21.15.69:0/4240614831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fdc0051c0 0x7f4fdc00b9c0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:51.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.819+0000 7f4ff9d9e640 1 -- 172.21.15.69:0/4240614831 >> 172.21.15.69:0/4240614831 conn(0x7f4ff409b7d0 msgr2=0x7f4ff409bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:51.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.819+0000 7f4ff9d9e640 1 -- 172.21.15.69:0/4240614831 shutdown_connections 2024-04-25T11:00:51.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.819+0000 7f4ff9d9e640 1 -- 172.21.15.69:0/4240614831 wait complete. 2024-04-25T11:00:51.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.820+0000 7f4ff9d9e640 1 Processor -- start 2024-04-25T11:00:51.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.821+0000 7f4ff9d9e640 1 -- start start 2024-04-25T11:00:51.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.822+0000 7f4ff9d9e640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4fdc0051c0 0x7f4ff41f69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:51.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.822+0000 7f4ff9d9e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ff4154fc0 0x7f4ff40c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:51.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.822+0000 7f4ff9d9e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ff40ce080 0x7f4ff40ce490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:51.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.822+0000 7f4ff9d9e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff40b2470 con 0x7f4ff4154fc0 2024-04-25T11:00:51.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.822+0000 7f4ff9d9e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff40d00f0 con 0x7f4ff40ce080 2024-04-25T11:00:51.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.822+0000 7f4ff9d9e640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f4ff40af850 con 0x7f4fdc0051c0 2024-04-25T11:00:51.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.822+0000 7f4ff2ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ff4154fc0 0x7f4ff40c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:51.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.822+0000 7f4ff2ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ff4154fc0 0x7f4ff40c8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48008/0 (socket says 172.21.15.69:48008) 2024-04-25T11:00:51.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.822+0000 7f4ff2ffd640 1 -- 172.21.15.69:0/3565617630 learned_addr learned my addr 172.21.15.69:0/3565617630 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:51.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.822+0000 7f4ff37fe640 1 --2- 172.21.15.69:0/3565617630 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4fdc0051c0 0x7f4ff41f69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:51.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.822+0000 7f4ff3fff640 1 --2- 172.21.15.69:0/3565617630 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ff40ce080 0x7f4ff40ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:51.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.822+0000 7f4ff2ffd640 1 -- 172.21.15.69:0/3565617630 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4fdc0051c0 msgr2=0x7f4ff41f69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.822+0000 7f4ff2ffd640 1 --2- 172.21.15.69:0/3565617630 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4fdc0051c0 0x7f4ff41f69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:51.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.822+0000 7f4ff2ffd640 1 -- 172.21.15.69:0/3565617630 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ff40ce080 msgr2=0x7f4ff40ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:51.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.822+0000 7f4ff2ffd640 1 --2- 172.21.15.69:0/3565617630 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ff40ce080 0x7f4ff40ce490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:51.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.822+0000 7f4ff2ffd640 1 -- 172.21.15.69:0/3565617630 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff40ce9d0 con 0x7f4ff4154fc0 2024-04-25T11:00:51.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.822+0000 7f4ff3fff640 1 --2- 172.21.15.69:0/3565617630 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ff40ce080 0x7f4ff40ce490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:51.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.822+0000 7f4ff37fe640 1 --2- 172.21.15.69:0/3565617630 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4fdc0051c0 0x7f4ff41f69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:51.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.822+0000 7f4ff2ffd640 1 --2- 172.21.15.69:0/3565617630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ff4154fc0 0x7f4ff40c8e60 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4fe80072c0 tx=0x7f4fe8007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:51.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.822+0000 7f4fdbfff640 1 -- 172.21.15.69:0/3565617630 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8001230 con 0x7f4ff4154fc0 2024-04-25T11:00:51.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.823+0000 7f4fdbfff640 1 -- 172.21.15.69:0/3565617630 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fe8018070 con 0x7f4ff4154fc0 2024-04-25T11:00:51.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.823+0000 7f4ff9d9e640 1 -- 172.21.15.69:0/3565617630 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ff41553d0 con 0x7f4ff4154fc0 2024-04-25T11:00:51.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.823+0000 7f4fdbfff640 1 -- 172.21.15.69:0/3565617630 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe800fc20 con 0x7f4ff4154fc0 2024-04-25T11:00:51.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.823+0000 7f4ff9d9e640 1 -- 172.21.15.69:0/3565617630 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ff40cad20 con 0x7f4ff4154fc0 2024-04-25T11:00:51.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.823+0000 7f4ff9d9e640 1 -- 172.21.15.69:0/3565617630 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ff4050740 con 0x7f4ff4154fc0 2024-04-25T11:00:51.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.824+0000 7f4fdbfff640 1 -- 172.21.15.69:0/3565617630 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4fe801d460 con 0x7f4ff4154fc0 2024-04-25T11:00:51.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.825+0000 7f4fdbfff640 1 --2- 172.21.15.69:0/3565617630 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4fe00744b0 0x7f4fe00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:51.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.825+0000 7f4ff37fe640 1 --2- 172.21.15.69:0/3565617630 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4fe00744b0 0x7f4fe00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:51.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.825+0000 7f4fdbfff640 1 -- 172.21.15.69:0/3565617630 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7f4fe8094fc0 con 0x7f4ff4154fc0 2024-04-25T11:00:51.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.825+0000 7f4ff37fe640 1 --2- 172.21.15.69:0/3565617630 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4fe00744b0 0x7f4fe00769c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4000c00 tx=0x7f4fe4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:51.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:51.829+0000 7f4fdbfff640 1 -- 172.21.15.69:0/3565617630 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4ff40ce9d0 con 0x7f4ff4154fc0 2024-04-25T11:00:52.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.004+0000 7f4ff9d9e640 1 -- 172.21.15.69:0/3565617630 --> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f4ff41e5a50 con 0x7f4fe00744b0 2024-04-25T11:00:52.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.004+0000 7f4fdbfff640 1 -- 172.21.15.69:0/3565617630 <== mgr.4107 v2:172.21.15.69:6832/1107421267 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f4ff41e5a50 con 0x7f4fe00744b0 2024-04-25T11:00:52.005 INFO:teuthology.orchestra.run.smithi069.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T11:00:52.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.006+0000 7f4ff9d9e640 1 -- 172.21.15.69:0/3565617630 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4fe00744b0 msgr2=0x7f4fe00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.006+0000 7f4ff9d9e640 1 --2- 172.21.15.69:0/3565617630 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4fe00744b0 0x7f4fe00769c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4000c00 tx=0x7f4fe4009040 comp rx=0 tx=0).stop 2024-04-25T11:00:52.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.006+0000 7f4ff9d9e640 1 -- 172.21.15.69:0/3565617630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ff4154fc0 msgr2=0x7f4ff40c8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.006+0000 7f4ff9d9e640 1 --2- 172.21.15.69:0/3565617630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ff4154fc0 0x7f4ff40c8e60 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4fe80072c0 tx=0x7f4fe8007780 comp rx=0 tx=0).stop 2024-04-25T11:00:52.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.006+0000 7f4ff9d9e640 1 -- 172.21.15.69:0/3565617630 shutdown_connections 2024-04-25T11:00:52.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.006+0000 7f4ff9d9e640 1 --2- 172.21.15.69:0/3565617630 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f4fe00744b0 0x7f4fe00769c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.006+0000 7f4ff9d9e640 1 --2- 172.21.15.69:0/3565617630 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ff40ce080 0x7f4ff40ce490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.006+0000 7f4ff9d9e640 1 --2- 172.21.15.69:0/3565617630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ff4154fc0 0x7f4ff40c8e60 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.006+0000 7f4ff9d9e640 1 --2- 172.21.15.69:0/3565617630 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4fdc0051c0 0x7f4ff41f69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.006+0000 7f4ff9d9e640 1 -- 172.21.15.69:0/3565617630 >> 172.21.15.69:0/3565617630 conn(0x7f4ff409b7d0 msgr2=0x7f4ff40b7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:52.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.006+0000 7f4ff9d9e640 1 -- 172.21.15.69:0/3565617630 shutdown_connections 2024-04-25T11:00:52.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.006+0000 7f4ff9d9e640 1 -- 172.21.15.69:0/3565617630 wait complete. 2024-04-25T11:00:52.018 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T11:00:52.018 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:00:52.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.129+0000 7f66c9297640 1 Processor -- start 2024-04-25T11:00:52.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.129+0000 7f66c9297640 1 -- start start 2024-04-25T11:00:52.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.131+0000 7f66c9297640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f66c4161710 0x7f66c415e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:52.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.131+0000 7f66c9297640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66c406c5c0 0x7f66c406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:52.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.131+0000 7f66c9297640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66c41609b0 con 0x7f66c4163fa0 2024-04-25T11:00:52.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.131+0000 7f66c9297640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f66c406cf10 con 0x7f66c4161710 2024-04-25T11:00:52.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.131+0000 7f66c9297640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f66c4164cd0 con 0x7f66c406c5c0 2024-04-25T11:00:52.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.131+0000 7f66c37fe640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7f66c4163fa0 0x7f66c41643b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:54486/0 (socket says 172.21.15.69:54486) 2024-04-25T11:00:52.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.131+0000 7f66c37fe640 1 -- 172.21.15.69:0/3317629791 learned_addr learned my addr 172.21.15.69:0/3317629791 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:52.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.131+0000 7f66c27fc640 1 --2- 172.21.15.69:0/3317629791 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66c406c5c0 0x7f66c406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:52.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.131+0000 7f66c2ffd640 1 --2- 172.21.15.69:0/3317629791 >> v2:172.21.15.146:3300/0 conn(0x7f66c4161710 0x7f66c415e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:52.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.131+0000 7f66c1ffb640 1 -- 172.21.15.69:0/3317629791 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1025120218 0 0) 0x7f66bc001110 con 0x7f66c4163fa0 2024-04-25T11:00:52.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.131+0000 7f66c1ffb640 1 -- 172.21.15.69:0/3317629791 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66ac003520 con 0x7f66c4163fa0 2024-04-25T11:00:52.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.131+0000 7f66c1ffb640 1 -- 172.21.15.69:0/3317629791 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3661772548 0 0) 0x7f66ac003520 con 0x7f66c4163fa0 2024-04-25T11:00:52.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.131+0000 7f66c2ffd640 1 -- 172.21.15.69:0/3317629791 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66c406c5c0 msgr2=0x7f66c406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:52.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.131+0000 7f66c2ffd640 1 --2- 172.21.15.69:0/3317629791 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66c406c5c0 0x7f66c406c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.131+0000 7f66c2ffd640 1 -- 172.21.15.69:0/3317629791 >> v1:172.21.15.69:6789/0 conn(0x7f66c4163fa0 legacy=0x7f66c41643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.131+0000 7f66c2ffd640 1 -- 172.21.15.69:0/3317629791 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c4160c30 con 0x7f66c4161710 2024-04-25T11:00:52.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.132+0000 7f66c2ffd640 1 --2- 172.21.15.69:0/3317629791 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66c4161710 0x7f66c415e080 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f66b4009d90 tx=0x7f66b402a560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b5369981525ed44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:52.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.132+0000 7f66c1ffb640 1 -- 172.21.15.69:0/3317629791 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc001840 con 0x7f66c4161710 2024-04-25T11:00:52.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.132+0000 7f66c1ffb640 1 -- 172.21.15.69:0/3317629791 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b4007540 con 0x7f66c4161710 2024-04-25T11:00:52.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.132+0000 7f66c1ffb640 1 -- 172.21.15.69:0/3317629791 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b40078f0 con 0x7f66c4161710 2024-04-25T11:00:52.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.133+0000 7f66c9297640 1 -- 172.21.15.69:0/3317629791 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66c4161710 msgr2=0x7f66c415e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.133+0000 7f66c9297640 1 --2- 172.21.15.69:0/3317629791 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66c4161710 0x7f66c415e080 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f66b4009d90 tx=0x7f66b402a560 comp rx=0 tx=0).stop 2024-04-25T11:00:52.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.133+0000 7f66c9297640 1 -- 172.21.15.69:0/3317629791 shutdown_connections 2024-04-25T11:00:52.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.133+0000 7f66c9297640 1 --2- 172.21.15.69:0/3317629791 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66c406c5c0 0x7f66c406c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.133+0000 7f66c9297640 1 --2- 172.21.15.69:0/3317629791 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66c4161710 0x7f66c415e080 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.133+0000 7f66c9297640 1 -- 172.21.15.69:0/3317629791 >> 172.21.15.69:0/3317629791 conn(0x7f66c40d5d90 msgr2=0x7f66c40d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:52.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.133+0000 7f66c9297640 1 -- 172.21.15.69:0/3317629791 shutdown_connections 2024-04-25T11:00:52.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.133+0000 7f66c9297640 1 -- 172.21.15.69:0/3317629791 wait complete. 2024-04-25T11:00:52.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.135+0000 7f66c9297640 1 Processor -- start 2024-04-25T11:00:52.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.135+0000 7f66c9297640 1 -- start start 2024-04-25T11:00:52.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.136+0000 7f66c9297640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66c406c5c0 0x7f66c415a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:52.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.136+0000 7f66c9297640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66c4161710 0x7f66c40c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:52.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.136+0000 7f66c9297640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66c4163fa0 0x7f66c40c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:52.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.136+0000 7f66c9297640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f66c40d49f0 con 0x7f66c406c5c0 2024-04-25T11:00:52.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.136+0000 7f66c9297640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f66c41611f0 con 0x7f66c4163fa0 2024-04-25T11:00:52.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.136+0000 7f66c9297640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f66c40d4b70 con 0x7f66c4161710 2024-04-25T11:00:52.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.136+0000 7f66c2ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66c406c5c0 0x7f66c415a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:52.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.136+0000 7f66c2ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66c406c5c0 0x7f66c415a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48016/0 (socket says 172.21.15.69:48016) 2024-04-25T11:00:52.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.136+0000 7f66c2ffd640 1 -- 172.21.15.69:0/1189000666 learned_addr learned my addr 172.21.15.69:0/1189000666 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:52.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.137+0000 7f66c27fc640 1 --2- 172.21.15.69:0/1189000666 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66c4161710 0x7f66c40c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:52.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.137+0000 7f66c37fe640 1 --2- 172.21.15.69:0/1189000666 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66c4163fa0 0x7f66c40c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:52.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.137+0000 7f66c2ffd640 1 -- 172.21.15.69:0/1189000666 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66c4161710 msgr2=0x7f66c40c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.137+0000 7f66c2ffd640 1 --2- 172.21.15.69:0/1189000666 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66c4161710 0x7f66c40c96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.137+0000 7f66c2ffd640 1 -- 172.21.15.69:0/1189000666 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66c4163fa0 msgr2=0x7f66c40c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.137+0000 7f66c2ffd640 1 --2- 172.21.15.69:0/1189000666 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66c4163fa0 0x7f66c40c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.137+0000 7f66c2ffd640 1 -- 172.21.15.69:0/1189000666 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c40cc7e0 con 0x7f66c406c5c0 2024-04-25T11:00:52.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.137+0000 7f66c2ffd640 1 --2- 172.21.15.69:0/1189000666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66c406c5c0 0x7f66c415a470 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f66b402ab50 tx=0x7f66b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:52.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.137+0000 7f66ab7fe640 1 -- 172.21.15.69:0/1189000666 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b4009450 con 0x7f66c406c5c0 2024-04-25T11:00:52.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.137+0000 7f66ab7fe640 1 -- 172.21.15.69:0/1189000666 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b4004240 con 0x7f66c406c5c0 2024-04-25T11:00:52.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.137+0000 7f66c9297640 1 -- 172.21.15.69:0/1189000666 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66c40ca7a0 con 0x7f66c406c5c0 2024-04-25T11:00:52.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.137+0000 7f66ab7fe640 1 -- 172.21.15.69:0/1189000666 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b40045f0 con 0x7f66c406c5c0 2024-04-25T11:00:52.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.137+0000 7f66c9297640 1 -- 172.21.15.69:0/1189000666 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66c40ad770 con 0x7f66c406c5c0 2024-04-25T11:00:52.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.138+0000 7f66c9297640 1 -- 172.21.15.69:0/1189000666 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f66c40cae60 con 0x7f66c406c5c0 2024-04-25T11:00:52.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.139+0000 7f66ab7fe640 1 -- 172.21.15.69:0/1189000666 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f66b403b410 con 0x7f66c406c5c0 2024-04-25T11:00:52.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.139+0000 7f66ab7fe640 1 --2- 172.21.15.69:0/1189000666 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f66b00744b0 0x7f66b00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:52.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.139+0000 7f66ab7fe640 1 -- 172.21.15.69:0/1189000666 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7f66b40b6450 con 0x7f66c406c5c0 2024-04-25T11:00:52.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.139+0000 7f66c27fc640 1 --2- 172.21.15.69:0/1189000666 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f66b00744b0 0x7f66b00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:52.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.139+0000 7f66ab7fe640 1 --2- 172.21.15.69:0/1189000666 >> [v2:172.21.15.146:6808/634688021,v1:172.21.15.146:6810/634688021] conn(0x7f66b007a460 0x7f66b007c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:52.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.139+0000 7f66ab7fe640 1 -- 172.21.15.69:0/1189000666 --> [v2:172.21.15.146:6808/634688021,v1:172.21.15.146:6810/634688021] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f66b4007540 con 0x7f66b007a460 2024-04-25T11:00:52.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.139+0000 7f66ab7fe640 1 -- 172.21.15.69:0/1189000666 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f66b40bd050 con 0x7f66c406c5c0 2024-04-25T11:00:52.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.140+0000 7f66c37fe640 1 --2- 172.21.15.69:0/1189000666 >> [v2:172.21.15.146:6808/634688021,v1:172.21.15.146:6810/634688021] conn(0x7f66b007a460 0x7f66b007c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:52.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.140+0000 7f66c27fc640 1 --2- 172.21.15.69:0/1189000666 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f66b00744b0 0x7f66b00769c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f66c40cbe90 tx=0x7f66b8009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:52.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.140+0000 7f66c37fe640 1 --2- 172.21.15.69:0/1189000666 >> [v2:172.21.15.146:6808/634688021,v1:172.21.15.146:6810/634688021] conn(0x7f66b007a460 0x7f66b007c8b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:52.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.141+0000 7f66ab7fe640 1 -- 172.21.15.69:0/1189000666 <== osd.6 v2:172.21.15.146:6808/634688021 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f66b4007540 con 0x7f66b007a460 2024-04-25T11:00:52.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.154+0000 7f66c9297640 1 -- 172.21.15.69:0/1189000666 --> [v2:172.21.15.146:6808/634688021,v1:172.21.15.146:6810/634688021] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f66c4050740 con 0x7f66b007a460 2024-04-25T11:00:52.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.154+0000 7f66ab7fe640 1 -- 172.21.15.69:0/1189000666 <== osd.6 v2:172.21.15.146:6808/634688021 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f66c4050740 con 0x7f66b007a460 2024-04-25T11:00:52.155 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T11:00:52.155 INFO:teuthology.orchestra.run.smithi069.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:00:52.155 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T11:00:52.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.154+0000 7f66c9297640 1 -- 172.21.15.69:0/1189000666 >> [v2:172.21.15.146:6808/634688021,v1:172.21.15.146:6810/634688021] conn(0x7f66b007a460 msgr2=0x7f66b007c8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.154+0000 7f66c9297640 1 --2- 172.21.15.69:0/1189000666 >> [v2:172.21.15.146:6808/634688021,v1:172.21.15.146:6810/634688021] conn(0x7f66b007a460 0x7f66b007c8b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.154+0000 7f66c9297640 1 -- 172.21.15.69:0/1189000666 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f66b00744b0 msgr2=0x7f66b00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.154+0000 7f66c9297640 1 --2- 172.21.15.69:0/1189000666 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f66b00744b0 0x7f66b00769c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f66c40cbe90 tx=0x7f66b8009190 comp rx=0 tx=0).stop 2024-04-25T11:00:52.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.154+0000 7f66c9297640 1 -- 172.21.15.69:0/1189000666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66c406c5c0 msgr2=0x7f66c415a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.154+0000 7f66c9297640 1 --2- 172.21.15.69:0/1189000666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66c406c5c0 0x7f66c415a470 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f66b402ab50 tx=0x7f66b4004a40 comp rx=0 tx=0).stop 2024-04-25T11:00:52.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.154+0000 7f66c37fe640 1 -- 172.21.15.69:0/1189000666 reap_dead start 2024-04-25T11:00:52.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.154+0000 7f66c9297640 1 -- 172.21.15.69:0/1189000666 shutdown_connections 2024-04-25T11:00:52.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.154+0000 7f66c9297640 1 -- 172.21.15.69:0/1189000666 >> 172.21.15.69:0/1189000666 conn(0x7f66c40d5d90 msgr2=0x7f66c415c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:52.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.155+0000 7f66c9297640 1 -- 172.21.15.69:0/1189000666 shutdown_connections 2024-04-25T11:00:52.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.155+0000 7f66c9297640 1 -- 172.21.15.69:0/1189000666 wait complete. 2024-04-25T11:00:52.166 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T11:00:52.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.273+0000 7efd8a300640 1 Processor -- start 2024-04-25T11:00:52.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.273+0000 7efd8a300640 1 -- start start 2024-04-25T11:00:52.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.274+0000 7efd8a300640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7efd840b1e80 0x7efd84169220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:52.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.274+0000 7efd8a300640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd840cf810 con 0x7efd840d15e0 2024-04-25T11:00:52.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.274+0000 7efd8a300640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd840b0690 con 0x7efd840b9270 2024-04-25T11:00:52.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.274+0000 7efd8a300640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7efd840d19f0 con 0x7efd840b1e80 2024-04-25T11:00:52.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.275+0000 7efd8889d640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7efd840b1e80 0x7efd84169220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:52.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.275+0000 7efd837fe640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7efd840d15e0 0x7efd84162600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:54496/0 (socket says 172.21.15.69:54496) 2024-04-25T11:00:52.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.275+0000 7efd8889d640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7efd840b1e80 0x7efd84169220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48020/0 (socket says 172.21.15.69:48020) 2024-04-25T11:00:52.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.275+0000 7efd837fe640 1 -- 172.21.15.69:0/1370718337 learned_addr learned my addr 172.21.15.69:0/1370718337 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:52.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.275+0000 7efd8889d640 1 -- 172.21.15.69:0/1370718337 >> v1:172.21.15.146:6789/0 conn(0x7efd840b9270 legacy=0x7efd84161ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.275+0000 7efd8889d640 1 -- 172.21.15.69:0/1370718337 >> v1:172.21.15.69:6789/0 conn(0x7efd840d15e0 legacy=0x7efd84162600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.275+0000 7efd8889d640 1 -- 172.21.15.69:0/1370718337 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd840cfab0 con 0x7efd840b1e80 2024-04-25T11:00:52.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.275+0000 7efd82ffd640 1 -- 172.21.15.69:0/1370718337 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 721052474 0 0) 0x7efd840cf810 con 0x7efd840d15e0 2024-04-25T11:00:52.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.276+0000 7efd8889d640 1 --2- 172.21.15.69:0/1370718337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd840b1e80 0x7efd84169220 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7efd7c009d90 tx=0x7efd7c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3242dd52c21086 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:52.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.276+0000 7efd82ffd640 1 -- 172.21.15.69:0/1370718337 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c031600 con 0x7efd840b1e80 2024-04-25T11:00:52.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.276+0000 7efd82ffd640 1 -- 172.21.15.69:0/1370718337 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd7c0317a0 con 0x7efd840b1e80 2024-04-25T11:00:52.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.276+0000 7efd82ffd640 1 -- 172.21.15.69:0/1370718337 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c031b70 con 0x7efd840b1e80 2024-04-25T11:00:52.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.276+0000 7efd8a300640 1 -- 172.21.15.69:0/1370718337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd840b1e80 msgr2=0x7efd84169220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.276+0000 7efd8a300640 1 --2- 172.21.15.69:0/1370718337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd840b1e80 0x7efd84169220 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7efd7c009d90 tx=0x7efd7c02a5d0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.276+0000 7efd8a300640 1 -- 172.21.15.69:0/1370718337 shutdown_connections 2024-04-25T11:00:52.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.276+0000 7efd8a300640 1 --2- 172.21.15.69:0/1370718337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd840b1e80 0x7efd84169220 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.276+0000 7efd8a300640 1 -- 172.21.15.69:0/1370718337 >> 172.21.15.69:0/1370718337 conn(0x7efd840d3aa0 msgr2=0x7efd840d3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:52.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.276+0000 7efd8a300640 1 -- 172.21.15.69:0/1370718337 shutdown_connections 2024-04-25T11:00:52.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.276+0000 7efd8a300640 1 -- 172.21.15.69:0/1370718337 wait complete. 2024-04-25T11:00:52.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.278+0000 7efd8a300640 1 Processor -- start 2024-04-25T11:00:52.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.278+0000 7efd8a300640 1 -- start start 2024-04-25T11:00:52.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.280+0000 7efd8a300640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd840b1e80 0x7efd841567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:52.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.280+0000 7efd8a300640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd840b9270 0x7efd8415fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:52.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.280+0000 7efd8a300640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd840d15e0 0x7efd841601d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:52.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.280+0000 7efd8a300640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efd84169760 con 0x7efd840b1e80 2024-04-25T11:00:52.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.280+0000 7efd8a300640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7efd84162d10 con 0x7efd840d15e0 2024-04-25T11:00:52.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.280+0000 7efd8a300640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7efd841698e0 con 0x7efd840b9270 2024-04-25T11:00:52.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.280+0000 7efd83fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd840b1e80 0x7efd841567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:52.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.280+0000 7efd83fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd840b1e80 0x7efd841567b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48026/0 (socket says 172.21.15.69:48026) 2024-04-25T11:00:52.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.280+0000 7efd83fff640 1 -- 172.21.15.69:0/3008000045 learned_addr learned my addr 172.21.15.69:0/3008000045 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:52.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.280+0000 7efd8889d640 1 --2- 172.21.15.69:0/3008000045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd840d15e0 0x7efd841601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:52.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.280+0000 7efd837fe640 1 --2- 172.21.15.69:0/3008000045 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd840b9270 0x7efd8415fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:52.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.280+0000 7efd83fff640 1 -- 172.21.15.69:0/3008000045 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd840b9270 msgr2=0x7efd8415fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.280+0000 7efd83fff640 1 --2- 172.21.15.69:0/3008000045 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd840b9270 0x7efd8415fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.280+0000 7efd83fff640 1 -- 172.21.15.69:0/3008000045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd840d15e0 msgr2=0x7efd841601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.280+0000 7efd83fff640 1 --2- 172.21.15.69:0/3008000045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd840d15e0 0x7efd841601d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.280+0000 7efd83fff640 1 -- 172.21.15.69:0/3008000045 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8415d860 con 0x7efd840b1e80 2024-04-25T11:00:52.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.280+0000 7efd8889d640 1 --2- 172.21.15.69:0/3008000045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd840d15e0 0x7efd841601d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:52.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.280+0000 7efd837fe640 1 --2- 172.21.15.69:0/3008000045 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd840b9270 0x7efd8415fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:52.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.280+0000 7efd83fff640 1 --2- 172.21.15.69:0/3008000045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd840b1e80 0x7efd841567b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7efd7400c430 tx=0x7efd7400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:52.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.280+0000 7efd80ff9640 1 -- 172.21.15.69:0/3008000045 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd74017070 con 0x7efd840b1e80 2024-04-25T11:00:52.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.280+0000 7efd80ff9640 1 -- 172.21.15.69:0/3008000045 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd740052d0 con 0x7efd840b1e80 2024-04-25T11:00:52.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.280+0000 7efd8a300640 1 -- 172.21.15.69:0/3008000045 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd8415db40 con 0x7efd840b1e80 2024-04-25T11:00:52.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.280+0000 7efd80ff9640 1 -- 172.21.15.69:0/3008000045 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd74003830 con 0x7efd840b1e80 2024-04-25T11:00:52.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.280+0000 7efd8a300640 1 -- 172.21.15.69:0/3008000045 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd840cae60 con 0x7efd840b1e80 2024-04-25T11:00:52.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.281+0000 7efd8a300640 1 -- 172.21.15.69:0/3008000045 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd84050740 con 0x7efd840b1e80 2024-04-25T11:00:52.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.282+0000 7efd80ff9640 1 -- 172.21.15.69:0/3008000045 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efd7401a070 con 0x7efd840b1e80 2024-04-25T11:00:52.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.282+0000 7efd80ff9640 1 --2- 172.21.15.69:0/3008000045 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7efd700744b0 0x7efd700769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:52.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.282+0000 7efd80ff9640 1 -- 172.21.15.69:0/3008000045 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7efd74094320 con 0x7efd840b1e80 2024-04-25T11:00:52.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.283+0000 7efd837fe640 1 --2- 172.21.15.69:0/3008000045 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7efd700744b0 0x7efd700769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:52.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.283+0000 7efd837fe640 1 --2- 172.21.15.69:0/3008000045 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7efd700744b0 0x7efd700769c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efd840d37d0 tx=0x7efd78009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:52.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.287+0000 7efd80ff9640 1 -- 172.21.15.69:0/3008000045 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efd7405ed50 con 0x7efd840b1e80 2024-04-25T11:00:52.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.461+0000 7efd8a300640 1 -- 172.21.15.69:0/3008000045 --> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7efd840d3440 con 0x7efd700744b0 2024-04-25T11:00:52.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.462+0000 7efd80ff9640 1 -- 172.21.15.69:0/3008000045 <== mgr.4107 v2:172.21.15.69:6832/1107421267 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7efd840d3440 con 0x7efd700744b0 2024-04-25T11:00:52.463 INFO:teuthology.orchestra.run.smithi069.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T11:00:52.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.464+0000 7efd8a300640 1 -- 172.21.15.69:0/3008000045 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7efd700744b0 msgr2=0x7efd700769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.464+0000 7efd8a300640 1 --2- 172.21.15.69:0/3008000045 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7efd700744b0 0x7efd700769c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efd840d37d0 tx=0x7efd78009040 comp rx=0 tx=0).stop 2024-04-25T11:00:52.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.464+0000 7efd8a300640 1 -- 172.21.15.69:0/3008000045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd840b1e80 msgr2=0x7efd841567b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.464+0000 7efd8a300640 1 --2- 172.21.15.69:0/3008000045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd840b1e80 0x7efd841567b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7efd7400c430 tx=0x7efd7400c8f0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.464+0000 7efd8a300640 1 -- 172.21.15.69:0/3008000045 shutdown_connections 2024-04-25T11:00:52.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.464+0000 7efd8a300640 1 --2- 172.21.15.69:0/3008000045 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7efd700744b0 0x7efd700769c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.464+0000 7efd8a300640 1 --2- 172.21.15.69:0/3008000045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd840d15e0 0x7efd841601d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.464+0000 7efd8a300640 1 --2- 172.21.15.69:0/3008000045 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd840b9270 0x7efd8415fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.464+0000 7efd8a300640 1 --2- 172.21.15.69:0/3008000045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd840b1e80 0x7efd841567b0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.464+0000 7efd8a300640 1 -- 172.21.15.69:0/3008000045 >> 172.21.15.69:0/3008000045 conn(0x7efd840d3aa0 msgr2=0x7efd840724f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:52.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.464+0000 7efd8a300640 1 -- 172.21.15.69:0/3008000045 shutdown_connections 2024-04-25T11:00:52.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.464+0000 7efd8a300640 1 -- 172.21.15.69:0/3008000045 wait complete. 2024-04-25T11:00:52.476 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T11:00:52.476 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:00:52.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.586+0000 7ff083b8c640 1 Processor -- start 2024-04-25T11:00:52.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.587+0000 7ff083b8c640 1 -- start start 2024-04-25T11:00:52.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.588+0000 7ff083b8c640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff07c161a50 0x7ff07c0b1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:52.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.588+0000 7ff083b8c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff07c06dd70 0x7ff07c0af570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:52.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.588+0000 7ff083b8c640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff07c0b1bd0 con 0x7ff07c0b17c0 2024-04-25T11:00:52.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.588+0000 7ff083b8c640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7ff07c0b3250 con 0x7ff07c161a50 2024-04-25T11:00:52.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.588+0000 7ff083b8c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff07c0b3530 con 0x7ff07c06dd70 2024-04-25T11:00:52.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.588+0000 7ff082129640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff07c06dd70 0x7ff07c0af570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:52.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.588+0000 7ff081928640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff07c161a50 0x7ff07c0b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:52.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.588+0000 7ff081127640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7ff07c0b17c0 0x7ff07c06d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.69:54008/0 (socket says 172.21.15.69:54008) 2024-04-25T11:00:52.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.588+0000 7ff081127640 1 -- 172.21.15.69:0/2564361771 learned_addr learned my addr 172.21.15.69:0/2564361771 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:52.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.589+0000 7ff081928640 1 -- 172.21.15.69:0/2564361771 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff07c06dd70 msgr2=0x7ff07c0af570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:52.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.589+0000 7ff081928640 1 --2- 172.21.15.69:0/2564361771 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff07c06dd70 0x7ff07c0af570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.589+0000 7ff081928640 1 -- 172.21.15.69:0/2564361771 >> v1:172.21.15.146:6789/0 conn(0x7ff07c0b17c0 legacy=0x7ff07c06d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.589+0000 7ff081928640 1 -- 172.21.15.69:0/2564361771 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff07c0b3810 con 0x7ff07c161a50 2024-04-25T11:00:52.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.589+0000 7ff080926640 1 -- 172.21.15.69:0/2564361771 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 280501303 0 0) 0x7ff06c001110 con 0x7ff07c0b17c0 2024-04-25T11:00:52.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.589+0000 7ff082129640 1 --2- 172.21.15.69:0/2564361771 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff07c06dd70 0x7ff07c0af570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:00:52.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.590+0000 7ff081928640 1 --2- 172.21.15.69:0/2564361771 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff07c161a50 0x7ff07c0b1280 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff078009d90 tx=0x7ff07802a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de5fb878431470ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:52.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.590+0000 7ff080926640 1 -- 172.21.15.69:0/2564361771 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff078031600 con 0x7ff07c161a50 2024-04-25T11:00:52.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.590+0000 7ff080926640 1 -- 172.21.15.69:0/2564361771 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0780317a0 con 0x7ff07c161a50 2024-04-25T11:00:52.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.590+0000 7ff080926640 1 -- 172.21.15.69:0/2564361771 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff078031b70 con 0x7ff07c161a50 2024-04-25T11:00:52.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.590+0000 7ff083b8c640 1 -- 172.21.15.69:0/2564361771 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff07c161a50 msgr2=0x7ff07c0b1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.590+0000 7ff083b8c640 1 --2- 172.21.15.69:0/2564361771 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff07c161a50 0x7ff07c0b1280 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff078009d90 tx=0x7ff07802a5d0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.590+0000 7ff083b8c640 1 -- 172.21.15.69:0/2564361771 shutdown_connections 2024-04-25T11:00:52.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.590+0000 7ff083b8c640 1 --2- 172.21.15.69:0/2564361771 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff07c06dd70 0x7ff07c0af570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.590+0000 7ff083b8c640 1 --2- 172.21.15.69:0/2564361771 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff07c161a50 0x7ff07c0b1280 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.590+0000 7ff083b8c640 1 -- 172.21.15.69:0/2564361771 >> 172.21.15.69:0/2564361771 conn(0x7ff07c09b7d0 msgr2=0x7ff07c0b70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:52.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.591+0000 7ff083b8c640 1 -- 172.21.15.69:0/2564361771 shutdown_connections 2024-04-25T11:00:52.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.591+0000 7ff083b8c640 1 -- 172.21.15.69:0/2564361771 wait complete. 2024-04-25T11:00:52.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.592+0000 7ff083b8c640 1 Processor -- start 2024-04-25T11:00:52.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.592+0000 7ff083b8c640 1 -- start start 2024-04-25T11:00:52.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.593+0000 7ff083b8c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff07c06dd70 0x7ff07c09e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:52.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.593+0000 7ff083b8c640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff07c0b17c0 0x7ff07c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:52.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.593+0000 7ff083b8c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff07c161a50 0x7ff07c149a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:52.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.594+0000 7ff083b8c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c161e60 con 0x7ff07c06dd70 2024-04-25T11:00:52.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.594+0000 7ff083b8c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c0af050 con 0x7ff07c161a50 2024-04-25T11:00:52.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.594+0000 7ff083b8c640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7ff07c0b2e70 con 0x7ff07c0b17c0 2024-04-25T11:00:52.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.594+0000 7ff081928640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff07c06dd70 0x7ff07c09e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:52.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.594+0000 7ff081928640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff07c06dd70 0x7ff07c09e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48038/0 (socket says 172.21.15.69:48038) 2024-04-25T11:00:52.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.594+0000 7ff081928640 1 -- 172.21.15.69:0/4105653211 learned_addr learned my addr 172.21.15.69:0/4105653211 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:52.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.594+0000 7ff081127640 1 --2- 172.21.15.69:0/4105653211 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff07c0b17c0 0x7ff07c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:52.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.594+0000 7ff082129640 1 --2- 172.21.15.69:0/4105653211 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff07c161a50 0x7ff07c149a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:52.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.594+0000 7ff081928640 1 -- 172.21.15.69:0/4105653211 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff07c0b17c0 msgr2=0x7ff07c070220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.594+0000 7ff081928640 1 --2- 172.21.15.69:0/4105653211 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff07c0b17c0 0x7ff07c070220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.594+0000 7ff081928640 1 -- 172.21.15.69:0/4105653211 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff07c161a50 msgr2=0x7ff07c149a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.594+0000 7ff081928640 1 --2- 172.21.15.69:0/4105653211 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff07c161a50 0x7ff07c149a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.594+0000 7ff081928640 1 -- 172.21.15.69:0/4105653211 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff07c14a3e0 con 0x7ff07c06dd70 2024-04-25T11:00:52.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.594+0000 7ff081928640 1 --2- 172.21.15.69:0/4105653211 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff07c06dd70 0x7ff07c09e160 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff078009d90 tx=0x7ff078004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:52.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.594+0000 7ff0727fc640 1 -- 172.21.15.69:0/4105653211 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07803f070 con 0x7ff07c06dd70 2024-04-25T11:00:52.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.594+0000 7ff0727fc640 1 -- 172.21.15.69:0/4105653211 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff078030be0 con 0x7ff07c06dd70 2024-04-25T11:00:52.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.594+0000 7ff0727fc640 1 -- 172.21.15.69:0/4105653211 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff078009450 con 0x7ff07c06dd70 2024-04-25T11:00:52.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.594+0000 7ff083b8c640 1 -- 172.21.15.69:0/4105653211 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff07c14a6c0 con 0x7ff07c06dd70 2024-04-25T11:00:52.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.594+0000 7ff083b8c640 1 -- 172.21.15.69:0/4105653211 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff07c14ac80 con 0x7ff07c06dd70 2024-04-25T11:00:52.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.595+0000 7ff083b8c640 1 -- 172.21.15.69:0/4105653211 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff07c050740 con 0x7ff07c06dd70 2024-04-25T11:00:52.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.596+0000 7ff0727fc640 1 -- 172.21.15.69:0/4105653211 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff078003990 con 0x7ff07c06dd70 2024-04-25T11:00:52.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.597+0000 7ff0727fc640 1 --2- 172.21.15.69:0/4105653211 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff0600744e0 0x7ff0600769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:52.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.597+0000 7ff0727fc640 1 -- 172.21.15.69:0/4105653211 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7ff0780b6210 con 0x7ff07c06dd70 2024-04-25T11:00:52.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.597+0000 7ff081127640 1 --2- 172.21.15.69:0/4105653211 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff0600744e0 0x7ff0600769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:52.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.597+0000 7ff0727fc640 1 --2- 172.21.15.69:0/4105653211 >> [v2:172.21.15.146:6824/3814215935,v1:172.21.15.146:6825/3814215935] conn(0x7ff06007a490 0x7ff06007c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:52.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.597+0000 7ff0727fc640 1 -- 172.21.15.69:0/4105653211 --> [v2:172.21.15.146:6824/3814215935,v1:172.21.15.146:6825/3814215935] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff078004850 con 0x7ff06007a490 2024-04-25T11:00:52.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.597+0000 7ff0727fc640 1 -- 172.21.15.69:0/4105653211 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff0780b66a0 con 0x7ff07c06dd70 2024-04-25T11:00:52.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.597+0000 7ff081127640 1 --2- 172.21.15.69:0/4105653211 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff0600744e0 0x7ff0600769f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff07c075ff0 tx=0x7ff06c007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:52.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.597+0000 7ff082129640 1 --2- 172.21.15.69:0/4105653211 >> [v2:172.21.15.146:6824/3814215935,v1:172.21.15.146:6825/3814215935] conn(0x7ff06007a490 0x7ff06007c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:52.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.598+0000 7ff082129640 1 --2- 172.21.15.69:0/4105653211 >> [v2:172.21.15.146:6824/3814215935,v1:172.21.15.146:6825/3814215935] conn(0x7ff06007a490 0x7ff06007c8e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:52.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.598+0000 7ff0727fc640 1 -- 172.21.15.69:0/4105653211 <== osd.7 v2:172.21.15.146:6824/3814215935 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff078004850 con 0x7ff06007a490 2024-04-25T11:00:52.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.611+0000 7ff083b8c640 1 -- 172.21.15.69:0/4105653211 --> [v2:172.21.15.146:6824/3814215935,v1:172.21.15.146:6825/3814215935] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff07c09e6a0 con 0x7ff06007a490 2024-04-25T11:00:52.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.612+0000 7ff0727fc640 1 -- 172.21.15.69:0/4105653211 <== osd.7 v2:172.21.15.146:6824/3814215935 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff07c09e6a0 con 0x7ff06007a490 2024-04-25T11:00:52.612 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T11:00:52.613 INFO:teuthology.orchestra.run.smithi069.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:00:52.613 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T11:00:52.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.612+0000 7ff083b8c640 1 -- 172.21.15.69:0/4105653211 >> [v2:172.21.15.146:6824/3814215935,v1:172.21.15.146:6825/3814215935] conn(0x7ff06007a490 msgr2=0x7ff06007c8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.612+0000 7ff083b8c640 1 --2- 172.21.15.69:0/4105653211 >> [v2:172.21.15.146:6824/3814215935,v1:172.21.15.146:6825/3814215935] conn(0x7ff06007a490 0x7ff06007c8e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.612+0000 7ff083b8c640 1 -- 172.21.15.69:0/4105653211 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff0600744e0 msgr2=0x7ff0600769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.612+0000 7ff083b8c640 1 --2- 172.21.15.69:0/4105653211 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7ff0600744e0 0x7ff0600769f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff07c075ff0 tx=0x7ff06c007de0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.612+0000 7ff083b8c640 1 -- 172.21.15.69:0/4105653211 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff07c06dd70 msgr2=0x7ff07c09e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.612+0000 7ff083b8c640 1 --2- 172.21.15.69:0/4105653211 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff07c06dd70 0x7ff07c09e160 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff078009d90 tx=0x7ff078004a40 comp rx=0 tx=0).stop 2024-04-25T11:00:52.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.612+0000 7ff082129640 1 -- 172.21.15.69:0/4105653211 reap_dead start 2024-04-25T11:00:52.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.612+0000 7ff083b8c640 1 -- 172.21.15.69:0/4105653211 shutdown_connections 2024-04-25T11:00:52.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.612+0000 7ff083b8c640 1 -- 172.21.15.69:0/4105653211 >> 172.21.15.69:0/4105653211 conn(0x7ff07c09b7d0 msgr2=0x7ff07c0afaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:52.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.612+0000 7ff083b8c640 1 -- 172.21.15.69:0/4105653211 shutdown_connections 2024-04-25T11:00:52.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.613+0000 7ff083b8c640 1 -- 172.21.15.69:0/4105653211 wait complete. 2024-04-25T11:00:52.624 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T11:00:52.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.730+0000 7fc2418a7640 1 Processor -- start 2024-04-25T11:00:52.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.730+0000 7fc2418a7640 1 -- start start 2024-04-25T11:00:52.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.732+0000 7fc2418a7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc23c0d4130 0x7fc23c0d1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:52.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.732+0000 7fc2418a7640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc23c06dcf0 0x7fc23c0d22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:52.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.732+0000 7fc2418a7640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc23c0b3c10 con 0x7fc23c06d8e0 2024-04-25T11:00:52.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.732+0000 7fc2418a7640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7fc23c0b9520 con 0x7fc23c06dcf0 2024-04-25T11:00:52.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.732+0000 7fc2418a7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc23c0b4820 con 0x7fc23c0d4130 2024-04-25T11:00:52.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.732+0000 7fc23affd640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fc23c06d8e0 0x7fc23c0d3a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:54498/0 (socket says 172.21.15.69:54498) 2024-04-25T11:00:52.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.732+0000 7fc23affd640 1 -- 172.21.15.69:0/1669127698 learned_addr learned my addr 172.21.15.69:0/1669127698 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:00:52.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.732+0000 7fc23a7fc640 1 --2- 172.21.15.69:0/1669127698 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc23c0d4130 0x7fc23c0d1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:52.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.732+0000 7fc23b7fe640 1 --2- 172.21.15.69:0/1669127698 >> v2:172.21.15.146:3300/0 conn(0x7fc23c06dcf0 0x7fc23c0d22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:52.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.732+0000 7fc239ffb640 1 -- 172.21.15.69:0/1669127698 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3172755160 0 0) 0x7fc23c0b3c10 con 0x7fc23c06d8e0 2024-04-25T11:00:52.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.732+0000 7fc239ffb640 1 -- 172.21.15.69:0/1669127698 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc224003520 con 0x7fc23c06d8e0 2024-04-25T11:00:52.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.732+0000 7fc23b7fe640 1 -- 172.21.15.69:0/1669127698 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc23c0d4130 msgr2=0x7fc23c0d1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:52.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.732+0000 7fc23b7fe640 1 --2- 172.21.15.69:0/1669127698 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc23c0d4130 0x7fc23c0d1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.732+0000 7fc23b7fe640 1 -- 172.21.15.69:0/1669127698 >> v1:172.21.15.69:6789/0 conn(0x7fc23c06d8e0 legacy=0x7fc23c0d3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.732+0000 7fc239ffb640 1 -- 172.21.15.69:0/1669127698 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3889823114 0 0) 0x7fc224003520 con 0x7fc23c06d8e0 2024-04-25T11:00:52.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.732+0000 7fc23b7fe640 1 -- 172.21.15.69:0/1669127698 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc23c0b3ef0 con 0x7fc23c06dcf0 2024-04-25T11:00:52.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.732+0000 7fc23a7fc640 1 --2- 172.21.15.69:0/1669127698 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc23c0d4130 0x7fc23c0d1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:00:52.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.733+0000 7fc23b7fe640 1 --2- 172.21.15.69:0/1669127698 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc23c06dcf0 0x7fc23c0d22a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc2340097a0 tx=0x7fc23402a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f43fb4bf3404258 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:52.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.733+0000 7fc239ffb640 1 -- 172.21.15.69:0/1669127698 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc234037070 con 0x7fc23c06dcf0 2024-04-25T11:00:52.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.733+0000 7fc239ffb640 1 -- 172.21.15.69:0/1669127698 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc23402ab10 con 0x7fc23c06dcf0 2024-04-25T11:00:52.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.733+0000 7fc239ffb640 1 -- 172.21.15.69:0/1669127698 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c0019f0 con 0x7fc23c06dcf0 2024-04-25T11:00:52.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.734+0000 7fc2418a7640 1 -- 172.21.15.69:0/1669127698 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc23c06dcf0 msgr2=0x7fc23c0d22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.734+0000 7fc2418a7640 1 --2- 172.21.15.69:0/1669127698 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc23c06dcf0 0x7fc23c0d22a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc2340097a0 tx=0x7fc23402a080 comp rx=0 tx=0).stop 2024-04-25T11:00:52.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.734+0000 7fc2418a7640 1 -- 172.21.15.69:0/1669127698 shutdown_connections 2024-04-25T11:00:52.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.734+0000 7fc2418a7640 1 --2- 172.21.15.69:0/1669127698 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc23c06dcf0 0x7fc23c0d22a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.734+0000 7fc2418a7640 1 --2- 172.21.15.69:0/1669127698 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc23c0d4130 0x7fc23c0d1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.734+0000 7fc2418a7640 1 -- 172.21.15.69:0/1669127698 >> 172.21.15.69:0/1669127698 conn(0x7fc23c08d110 msgr2=0x7fc23c08d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:52.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.734+0000 7fc2418a7640 1 -- 172.21.15.69:0/1669127698 shutdown_connections 2024-04-25T11:00:52.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.734+0000 7fc2418a7640 1 -- 172.21.15.69:0/1669127698 wait complete. 2024-04-25T11:00:52.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.736+0000 7fc2418a7640 1 Processor -- start 2024-04-25T11:00:52.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.736+0000 7fc2418a7640 1 -- start start 2024-04-25T11:00:52.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.737+0000 7fc2418a7640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc23c06d8e0 0x7fc23c15c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:52.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.737+0000 7fc2418a7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc23c06dcf0 0x7fc23c09d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:52.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.737+0000 7fc2418a7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc23c09dab0 0x7fc23c1f5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:52.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.737+0000 7fc2418a7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c0b38e0 con 0x7fc23c06dcf0 2024-04-25T11:00:52.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.737+0000 7fc2418a7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c0b44b0 con 0x7fc23c09dab0 2024-04-25T11:00:52.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.737+0000 7fc2418a7640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc23c0d27e0 con 0x7fc23c06d8e0 2024-04-25T11:00:52.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.737+0000 7fc23a7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc23c06dcf0 0x7fc23c09d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:52.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.737+0000 7fc23a7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc23c06dcf0 0x7fc23c09d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48048/0 (socket says 172.21.15.69:48048) 2024-04-25T11:00:52.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.737+0000 7fc23a7fc640 1 -- 172.21.15.69:0/3169451571 learned_addr learned my addr 172.21.15.69:0/3169451571 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:52.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.738+0000 7fc23b7fe640 1 --2- 172.21.15.69:0/3169451571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc23c09dab0 0x7fc23c1f5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:52.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.738+0000 7fc23affd640 1 --2- 172.21.15.69:0/3169451571 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc23c06d8e0 0x7fc23c15c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:52.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.738+0000 7fc23a7fc640 1 -- 172.21.15.69:0/3169451571 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc23c06d8e0 msgr2=0x7fc23c15c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.738+0000 7fc23a7fc640 1 --2- 172.21.15.69:0/3169451571 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc23c06d8e0 0x7fc23c15c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.738+0000 7fc23a7fc640 1 -- 172.21.15.69:0/3169451571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc23c09dab0 msgr2=0x7fc23c1f5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.738+0000 7fc23a7fc640 1 --2- 172.21.15.69:0/3169451571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc23c09dab0 0x7fc23c1f5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.738+0000 7fc23a7fc640 1 -- 172.21.15.69:0/3169451571 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc23c15cd50 con 0x7fc23c06dcf0 2024-04-25T11:00:52.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.738+0000 7fc23affd640 1 --2- 172.21.15.69:0/3169451571 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc23c06d8e0 0x7fc23c15c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:00:52.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.738+0000 7fc23a7fc640 1 --2- 172.21.15.69:0/3169451571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc23c06dcf0 0x7fc23c09d570 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc23000e960 tx=0x7fc23000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:52.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.738+0000 7fc223fff640 1 -- 172.21.15.69:0/3169451571 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc230017070 con 0x7fc23c06dcf0 2024-04-25T11:00:52.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.738+0000 7fc223fff640 1 -- 172.21.15.69:0/3169451571 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc23000ca00 con 0x7fc23c06dcf0 2024-04-25T11:00:52.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.738+0000 7fc2418a7640 1 -- 172.21.15.69:0/3169451571 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc23c0b8110 con 0x7fc23c06dcf0 2024-04-25T11:00:52.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.738+0000 7fc223fff640 1 -- 172.21.15.69:0/3169451571 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23000cdd0 con 0x7fc23c06dcf0 2024-04-25T11:00:52.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.738+0000 7fc2418a7640 1 -- 172.21.15.69:0/3169451571 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc23c09e520 con 0x7fc23c06dcf0 2024-04-25T11:00:52.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.739+0000 7fc2418a7640 1 -- 172.21.15.69:0/3169451571 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc23c0b7e80 con 0x7fc23c06dcf0 2024-04-25T11:00:52.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.740+0000 7fc223fff640 1 -- 172.21.15.69:0/3169451571 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc23001a070 con 0x7fc23c06dcf0 2024-04-25T11:00:52.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.740+0000 7fc223fff640 1 --2- 172.21.15.69:0/3169451571 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc2280744b0 0x7fc2280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:52.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.740+0000 7fc223fff640 1 -- 172.21.15.69:0/3169451571 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7fc230094db0 con 0x7fc23c06dcf0 2024-04-25T11:00:52.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.740+0000 7fc23affd640 1 --2- 172.21.15.69:0/3169451571 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc2280744b0 0x7fc2280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:52.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.741+0000 7fc23affd640 1 --2- 172.21.15.69:0/3169451571 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc2280744b0 0x7fc2280769c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc22c005c00 tx=0x7fc22c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:52.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.745+0000 7fc223fff640 1 -- 172.21.15.69:0/3169451571 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc23005f190 con 0x7fc23c06dcf0 2024-04-25T11:00:52.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.921+0000 7fc2418a7640 1 -- 172.21.15.69:0/3169451571 --> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fc23c050740 con 0x7fc2280744b0 2024-04-25T11:00:52.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.921+0000 7fc223fff640 1 -- 172.21.15.69:0/3169451571 <== mgr.4107 v2:172.21.15.69:6832/1107421267 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc23c050740 con 0x7fc2280744b0 2024-04-25T11:00:52.921 INFO:teuthology.orchestra.run.smithi069.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T11:00:52.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.923+0000 7fc2418a7640 1 -- 172.21.15.69:0/3169451571 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc2280744b0 msgr2=0x7fc2280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.923+0000 7fc2418a7640 1 --2- 172.21.15.69:0/3169451571 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc2280744b0 0x7fc2280769c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc22c005c00 tx=0x7fc22c009040 comp rx=0 tx=0).stop 2024-04-25T11:00:52.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.923+0000 7fc2418a7640 1 -- 172.21.15.69:0/3169451571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc23c06dcf0 msgr2=0x7fc23c09d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:52.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.923+0000 7fc2418a7640 1 --2- 172.21.15.69:0/3169451571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc23c06dcf0 0x7fc23c09d570 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc23000e960 tx=0x7fc23000ee20 comp rx=0 tx=0).stop 2024-04-25T11:00:52.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.923+0000 7fc2418a7640 1 -- 172.21.15.69:0/3169451571 shutdown_connections 2024-04-25T11:00:52.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.923+0000 7fc2418a7640 1 --2- 172.21.15.69:0/3169451571 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc2280744b0 0x7fc2280769c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.923+0000 7fc2418a7640 1 --2- 172.21.15.69:0/3169451571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc23c09dab0 0x7fc23c1f5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.923+0000 7fc2418a7640 1 --2- 172.21.15.69:0/3169451571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc23c06dcf0 0x7fc23c09d570 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.923+0000 7fc2418a7640 1 --2- 172.21.15.69:0/3169451571 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc23c06d8e0 0x7fc23c15c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:52.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.923+0000 7fc2418a7640 1 -- 172.21.15.69:0/3169451571 >> 172.21.15.69:0/3169451571 conn(0x7fc23c08d110 msgr2=0x7fc23c0d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:52.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.923+0000 7fc2418a7640 1 -- 172.21.15.69:0/3169451571 shutdown_connections 2024-04-25T11:00:52.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:52.923+0000 7fc2418a7640 1 -- 172.21.15.69:0/3169451571 wait complete. 2024-04-25T11:00:52.935 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:00:53.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.041+0000 7fefa731f640 1 Processor -- start 2024-04-25T11:00:53.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.041+0000 7fefa731f640 1 -- start start 2024-04-25T11:00:53.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.042+0000 7fefa731f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fefa006ccd0 0x7fefa00d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:53.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.042+0000 7fefa731f640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fefa00d4a50 0x7fefa00d4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:53.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.042+0000 7fefa731f640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefa00d1850 con 0x7fefa00d0e00 2024-04-25T11:00:53.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.042+0000 7fefa731f640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7fefa0160de0 con 0x7fefa00d4a50 2024-04-25T11:00:53.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.042+0000 7fefa731f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fefa01610c0 con 0x7fefa006ccd0 2024-04-25T11:00:53.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.042+0000 7fefa58bc640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fefa00d4a50 0x7fefa00d4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:53.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.042+0000 7fefa58bc640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7fefa00d4a50 0x7fefa00d4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48054/0 (socket says 172.21.15.69:48054) 2024-04-25T11:00:53.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.042+0000 7fefa50bb640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fefa00d0e00 0x7fefa006c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:54500/0 (socket says 172.21.15.69:54500) 2024-04-25T11:00:53.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.042+0000 7fefa58bc640 1 -- 172.21.15.69:0/3831301196 learned_addr learned my addr 172.21.15.69:0/3831301196 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:53.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.043+0000 7fefa48ba640 1 --2- 172.21.15.69:0/3831301196 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fefa006ccd0 0x7fefa00d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:53.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.043+0000 7fefa58bc640 1 -- 172.21.15.69:0/3831301196 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fefa006ccd0 msgr2=0x7fefa00d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:00:53.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.043+0000 7fefa58bc640 1 --2- 172.21.15.69:0/3831301196 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fefa006ccd0 0x7fefa00d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:53.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.043+0000 7fefa58bc640 1 -- 172.21.15.69:0/3831301196 >> v1:172.21.15.69:6789/0 conn(0x7fefa00d0e00 legacy=0x7fefa006c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:53.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.043+0000 7fefa58bc640 1 -- 172.21.15.69:0/3831301196 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa01613a0 con 0x7fefa00d4a50 2024-04-25T11:00:53.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.043+0000 7fefa58bc640 1 --2- 172.21.15.69:0/3831301196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fefa00d4a50 0x7fefa00d4e60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fef98009e50 tx=0x7fef9802a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c6e475aa66e0d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:53.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.044+0000 7fef97fff640 1 -- 172.21.15.69:0/3831301196 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef98036070 con 0x7fefa00d4a50 2024-04-25T11:00:53.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.044+0000 7fef97fff640 1 -- 172.21.15.69:0/3831301196 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef98031600 con 0x7fefa00d4a50 2024-04-25T11:00:53.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.044+0000 7fef97fff640 1 -- 172.21.15.69:0/3831301196 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef980319d0 con 0x7fefa00d4a50 2024-04-25T11:00:53.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.044+0000 7fefa731f640 1 -- 172.21.15.69:0/3831301196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fefa00d4a50 msgr2=0x7fefa00d4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:53.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.044+0000 7fefa731f640 1 --2- 172.21.15.69:0/3831301196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fefa00d4a50 0x7fefa00d4e60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fef98009e50 tx=0x7fef9802a630 comp rx=0 tx=0).stop 2024-04-25T11:00:53.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.044+0000 7fefa731f640 1 -- 172.21.15.69:0/3831301196 shutdown_connections 2024-04-25T11:00:53.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.044+0000 7fefa731f640 1 --2- 172.21.15.69:0/3831301196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fefa00d4a50 0x7fefa00d4e60 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:53.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.044+0000 7fefa731f640 1 --2- 172.21.15.69:0/3831301196 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fefa006ccd0 0x7fefa00d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:53.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.044+0000 7fefa731f640 1 -- 172.21.15.69:0/3831301196 >> 172.21.15.69:0/3831301196 conn(0x7fefa00d5d90 msgr2=0x7fefa00d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:53.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.044+0000 7fefa731f640 1 -- 172.21.15.69:0/3831301196 shutdown_connections 2024-04-25T11:00:53.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.044+0000 7fefa731f640 1 -- 172.21.15.69:0/3831301196 wait complete. 2024-04-25T11:00:53.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.046+0000 7fefa731f640 1 Processor -- start 2024-04-25T11:00:53.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.046+0000 7fefa731f640 1 -- start start 2024-04-25T11:00:53.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.047+0000 7fefa731f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefa00d0e00 0x7fefa0154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:53.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.047+0000 7fefa731f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fefa00d4a50 0x7fefa00ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:53.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.047+0000 7fefa731f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fefa00cd310 0x7fefa00ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:53.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.047+0000 7fefa731f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fefa0164e00 con 0x7fefa00d4a50 2024-04-25T11:00:53.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.047+0000 7fefa731f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fefa00b5320 con 0x7fefa00d0e00 2024-04-25T11:00:53.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.047+0000 7fefa731f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fefa0164f80 con 0x7fefa00cd310 2024-04-25T11:00:53.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.047+0000 7fefa48ba640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fefa00d4a50 0x7fefa00ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:53.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.047+0000 7fefa48ba640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fefa00d4a50 0x7fefa00ccdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48064/0 (socket says 172.21.15.69:48064) 2024-04-25T11:00:53.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.047+0000 7fefa48ba640 1 -- 172.21.15.69:0/635045958 learned_addr learned my addr 172.21.15.69:0/635045958 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:00:53.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.048+0000 7fefa58bc640 1 --2- 172.21.15.69:0/635045958 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fefa00cd310 0x7fefa00ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:53.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.048+0000 7fefa50bb640 1 --2- 172.21.15.69:0/635045958 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefa00d0e00 0x7fefa0154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:53.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.048+0000 7fefa48ba640 1 -- 172.21.15.69:0/635045958 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fefa00cd310 msgr2=0x7fefa00ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:53.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.048+0000 7fefa48ba640 1 --2- 172.21.15.69:0/635045958 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fefa00cd310 0x7fefa00ce4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:53.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.048+0000 7fefa48ba640 1 -- 172.21.15.69:0/635045958 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefa00d0e00 msgr2=0x7fefa0154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:53.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.048+0000 7fefa48ba640 1 --2- 172.21.15.69:0/635045958 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefa00d0e00 0x7fefa0154110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:53.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.048+0000 7fefa48ba640 1 -- 172.21.15.69:0/635045958 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa0154650 con 0x7fefa00d4a50 2024-04-25T11:00:53.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.048+0000 7fefa48ba640 1 --2- 172.21.15.69:0/635045958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fefa00d4a50 0x7fefa00ccdd0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fef9000d5f0 tx=0x7fef9000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:53.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.048+0000 7fef95ffb640 1 -- 172.21.15.69:0/635045958 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef900125f0 con 0x7fefa00d4a50 2024-04-25T11:00:53.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.048+0000 7fef95ffb640 1 -- 172.21.15.69:0/635045958 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef90019070 con 0x7fefa00d4a50 2024-04-25T11:00:53.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.048+0000 7fefa731f640 1 -- 172.21.15.69:0/635045958 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa00cdde0 con 0x7fefa00d4a50 2024-04-25T11:00:53.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.048+0000 7fef95ffb640 1 -- 172.21.15.69:0/635045958 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef900129d0 con 0x7fefa00d4a50 2024-04-25T11:00:53.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.048+0000 7fefa731f640 1 -- 172.21.15.69:0/635045958 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa00cdb00 con 0x7fefa00d4a50 2024-04-25T11:00:53.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.049+0000 7fefa731f640 1 -- 172.21.15.69:0/635045958 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefa0050740 con 0x7fefa00d4a50 2024-04-25T11:00:53.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.050+0000 7fef95ffb640 1 -- 172.21.15.69:0/635045958 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fef90012b70 con 0x7fefa00d4a50 2024-04-25T11:00:53.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.050+0000 7fef95ffb640 1 --2- 172.21.15.69:0/635045958 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fef840744e0 0x7fef840769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:00:53.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.050+0000 7fef95ffb640 1 -- 172.21.15.69:0/635045958 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7fef90095390 con 0x7fefa00d4a50 2024-04-25T11:00:53.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.050+0000 7fefa50bb640 1 --2- 172.21.15.69:0/635045958 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fef840744e0 0x7fef840769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:00:53.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.051+0000 7fefa50bb640 1 --2- 172.21.15.69:0/635045958 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fef840744e0 0x7fef840769f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fef9c000f60 tx=0x7fef9c007c60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:00:53.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.055+0000 7fef95ffb640 1 -- 172.21.15.69:0/635045958 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fef9005fdc0 con 0x7fefa00d4a50 2024-04-25T11:00:53.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.226+0000 7fefa731f640 1 -- 172.21.15.69:0/635045958 --> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fefa0160820 con 0x7fef840744e0 2024-04-25T11:00:53.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.227+0000 7fef95ffb640 1 -- 172.21.15.69:0/635045958 <== mgr.4107 v2:172.21.15.69:6832/1107421267 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13334 (secure 0 0 0) 0x7fefa0160820 con 0x7fef840744e0 2024-04-25T11:00:53.228 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:00:53.228 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-04-25T11:00:53.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.229+0000 7fefa731f640 1 -- 172.21.15.69:0/635045958 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fef840744e0 msgr2=0x7fef840769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:53.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.229+0000 7fefa731f640 1 --2- 172.21.15.69:0/635045958 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fef840744e0 0x7fef840769f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fef9c000f60 tx=0x7fef9c007c60 comp rx=0 tx=0).stop 2024-04-25T11:00:53.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.229+0000 7fefa731f640 1 -- 172.21.15.69:0/635045958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fefa00d4a50 msgr2=0x7fefa00ccdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:00:53.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.229+0000 7fefa731f640 1 --2- 172.21.15.69:0/635045958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fefa00d4a50 0x7fefa00ccdd0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fef9000d5f0 tx=0x7fef9000dab0 comp rx=0 tx=0).stop 2024-04-25T11:00:53.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.229+0000 7fefa731f640 1 -- 172.21.15.69:0/635045958 shutdown_connections 2024-04-25T11:00:53.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.229+0000 7fefa731f640 1 --2- 172.21.15.69:0/635045958 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fef840744e0 0x7fef840769f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:53.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.229+0000 7fefa731f640 1 --2- 172.21.15.69:0/635045958 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fefa00cd310 0x7fefa00ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:53.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.229+0000 7fefa731f640 1 --2- 172.21.15.69:0/635045958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fefa00d4a50 0x7fefa00ccdd0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:53.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.229+0000 7fefa731f640 1 --2- 172.21.15.69:0/635045958 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefa00d0e00 0x7fefa0154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:00:53.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.229+0000 7fefa731f640 1 -- 172.21.15.69:0/635045958 >> 172.21.15.69:0/635045958 conn(0x7fefa00d5d90 msgr2=0x7fefa006c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:00:53.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.229+0000 7fefa731f640 1 -- 172.21.15.69:0/635045958 shutdown_connections 2024-04-25T11:00:53.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:00:53.229+0000 7fefa731f640 1 -- 172.21.15.69:0/635045958 wait complete. 2024-04-25T11:00:53.240 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2024-04-25T11:00:52.424240+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220512,"kb_used_data":4328,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754208,"statfs":{"total":773094113280,"available":772868308992,"internally_reserved":0,"allocated":4431872,"data_stored":2448240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.292598"},"pg_stats":[{"pgid":"1.0","version":"15'72","reported_seq":137,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:00:45.309701+0000","last_change":"2024-04-25T10:59:07.839466+0000","last_active":"2024-04-25T11:00:45.309701+0000","last_peered":"2024-04-25T11:00:45.309701+0000","last_clean":"2024-04-25T11:00:45.309701+0000","last_became_active":"2024-04-25T10:59:07.836269+0000","last_became_peered":"2024-04-25T10:59:07.836269+0000","last_unstale":"2024-04-25T11:00:45.309701+0000","last_undegraded":"2024-04-25T11:00:45.309701+0000","last_fullsized":"2024-04-25T11:00:45.309701+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:59:06.549663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:59:06.549663+0000","last_clean_scrub_stamp":"2024-04-25T10:59:06.549663+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:21.575330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":999424,"data_stored":748806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":158438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":158438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":158438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":158438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":158438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":158438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":999424,"data_stored":748806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:00:53.244 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T10:59:06.549663+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=59, tm_sec=6, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=0, tm_sec=48, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:00:53.244 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T11:01:13.245 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:01:13.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.361+0000 7fabab0aa640 1 Processor -- start 2024-04-25T11:01:13.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.361+0000 7fabab0aa640 1 -- start start 2024-04-25T11:01:13.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.362+0000 7fabab0aa640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7faba4143440 0x7faba4160b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:13.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.362+0000 7fabab0aa640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7faba41610a0 0x7faba40d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:13.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.362+0000 7fabab0aa640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faba41614b0 con 0x7faba40d0830 2024-04-25T11:01:13.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.362+0000 7fabab0aa640 1 -- --> v2:172.21.15.69:3300/0 -- mon_getmap magic: 0 -- 0x7faba406f4a0 con 0x7faba41610a0 2024-04-25T11:01:13.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.362+0000 7fabab0aa640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7faba4070940 con 0x7faba4143440 2024-04-25T11:01:13.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.362+0000 7faba3fff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7faba41610a0 0x7faba40d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:13.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.362+0000 7faba3fff640 1 --2- >> v2:172.21.15.69:3300/0 conn(0x7faba41610a0 0x7faba40d02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41302/0 (socket says 172.21.15.69:41302) 2024-04-25T11:01:13.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.362+0000 7faba3fff640 1 -- 172.21.15.69:0/1437268861 learned_addr learned my addr 172.21.15.69:0/1437268861 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:01:13.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.363+0000 7faba8e46640 1 --2- 172.21.15.69:0/1437268861 >> v2:172.21.15.146:3300/0 conn(0x7faba4143440 0x7faba4160b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:13.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.363+0000 7faba3fff640 1 -- 172.21.15.69:0/1437268861 >> v2:172.21.15.146:3300/0 conn(0x7faba4143440 msgr2=0x7faba4160b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:13.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.363+0000 7faba3fff640 1 --2- 172.21.15.69:0/1437268861 >> v2:172.21.15.146:3300/0 conn(0x7faba4143440 0x7faba4160b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:13.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.363+0000 7faba3fff640 1 -- 172.21.15.69:0/1437268861 >> v1:172.21.15.146:6789/0 conn(0x7faba40d0830 legacy=0x7faba415d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:13.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.363+0000 7faba3fff640 1 -- 172.21.15.69:0/1437268861 --> v2:172.21.15.69:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba4143850 con 0x7faba41610a0 2024-04-25T11:01:13.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.363+0000 7faba3fff640 1 --2- 172.21.15.69:0/1437268861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faba41610a0 0x7faba40d02f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fab8c00bb10 tx=0x7fab8c02c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd74ce8c8979703b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:13.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.364+0000 7faba37fe640 1 -- 172.21.15.69:0/1437268861 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c032790 con 0x7faba41610a0 2024-04-25T11:01:13.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.364+0000 7faba37fe640 1 -- 172.21.15.69:0/1437268861 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab8c038070 con 0x7faba41610a0 2024-04-25T11:01:13.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.364+0000 7faba37fe640 1 -- 172.21.15.69:0/1437268861 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c032b50 con 0x7faba41610a0 2024-04-25T11:01:13.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.364+0000 7fabab0aa640 1 -- 172.21.15.69:0/1437268861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faba41610a0 msgr2=0x7faba40d02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:13.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.364+0000 7fabab0aa640 1 --2- 172.21.15.69:0/1437268861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faba41610a0 0x7faba40d02f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fab8c00bb10 tx=0x7fab8c02c8d0 comp rx=0 tx=0).stop 2024-04-25T11:01:13.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.364+0000 7fabab0aa640 1 -- 172.21.15.69:0/1437268861 shutdown_connections 2024-04-25T11:01:13.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.364+0000 7fabab0aa640 1 --2- 172.21.15.69:0/1437268861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faba41610a0 0x7faba40d02f0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:13.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.364+0000 7fabab0aa640 1 --2- 172.21.15.69:0/1437268861 >> v2:172.21.15.146:3300/0 conn(0x7faba4143440 0x7faba4160b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:13.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.364+0000 7fabab0aa640 1 -- 172.21.15.69:0/1437268861 >> 172.21.15.69:0/1437268861 conn(0x7faba40d1ae0 msgr2=0x7faba40d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:13.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.364+0000 7fabab0aa640 1 -- 172.21.15.69:0/1437268861 shutdown_connections 2024-04-25T11:01:13.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.364+0000 7fabab0aa640 1 -- 172.21.15.69:0/1437268861 wait complete. 2024-04-25T11:01:13.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.366+0000 7fabab0aa640 1 Processor -- start 2024-04-25T11:01:13.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.366+0000 7fabab0aa640 1 -- start start 2024-04-25T11:01:13.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.368+0000 7fabab0aa640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faba40d0830 0x7faba413f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:13.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.368+0000 7fabab0aa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faba4143440 0x7faba40cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:13.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.368+0000 7fabab0aa640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faba41610a0 0x7faba40c7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:13.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.368+0000 7fabab0aa640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7faba406f130 con 0x7faba4143440 2024-04-25T11:01:13.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.368+0000 7fabab0aa640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7faba40d1400 con 0x7faba40d0830 2024-04-25T11:01:13.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.368+0000 7fabab0aa640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7faba40b4fb0 con 0x7faba41610a0 2024-04-25T11:01:13.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.368+0000 7faba3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faba4143440 0x7faba40cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:13.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.368+0000 7faba3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faba4143440 0x7faba40cdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41312/0 (socket says 172.21.15.69:41312) 2024-04-25T11:01:13.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.368+0000 7faba3fff640 1 -- 172.21.15.69:0/3237452239 learned_addr learned my addr 172.21.15.69:0/3237452239 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:01:13.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.368+0000 7faba8e46640 1 --2- 172.21.15.69:0/3237452239 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faba40d0830 0x7faba413f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:13.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.368+0000 7faba9647640 1 --2- 172.21.15.69:0/3237452239 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faba41610a0 0x7faba40c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:13.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.368+0000 7faba3fff640 1 -- 172.21.15.69:0/3237452239 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faba41610a0 msgr2=0x7faba40c7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:13.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.368+0000 7faba3fff640 1 --2- 172.21.15.69:0/3237452239 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faba41610a0 0x7faba40c7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:13.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.368+0000 7faba3fff640 1 -- 172.21.15.69:0/3237452239 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faba40d0830 msgr2=0x7faba413f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:13.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.368+0000 7faba3fff640 1 --2- 172.21.15.69:0/3237452239 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faba40d0830 0x7faba413f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:13.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.368+0000 7faba3fff640 1 -- 172.21.15.69:0/3237452239 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba413f5f0 con 0x7faba4143440 2024-04-25T11:01:13.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.368+0000 7faba9647640 1 --2- 172.21.15.69:0/3237452239 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faba41610a0 0x7faba40c7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:01:13.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.368+0000 7faba3fff640 1 --2- 172.21.15.69:0/3237452239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faba4143440 0x7faba40cdab0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fab8c000c00 tx=0x7fab8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:13.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.368+0000 7faba17fa640 1 -- 172.21.15.69:0/3237452239 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c033620 con 0x7faba4143440 2024-04-25T11:01:13.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.368+0000 7faba17fa640 1 -- 172.21.15.69:0/3237452239 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab8c0337e0 con 0x7faba4143440 2024-04-25T11:01:13.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.368+0000 7faba17fa640 1 -- 172.21.15.69:0/3237452239 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c033bd0 con 0x7faba4143440 2024-04-25T11:01:13.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.368+0000 7fabab0aa640 1 -- 172.21.15.69:0/3237452239 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faba40cca10 con 0x7faba4143440 2024-04-25T11:01:13.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.368+0000 7fabab0aa640 1 -- 172.21.15.69:0/3237452239 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faba40ccfd0 con 0x7faba4143440 2024-04-25T11:01:13.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.370+0000 7fabab0aa640 1 -- 172.21.15.69:0/3237452239 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faba4050740 con 0x7faba4143440 2024-04-25T11:01:13.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.370+0000 7faba17fa640 1 -- 172.21.15.69:0/3237452239 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fab8c04d060 con 0x7faba4143440 2024-04-25T11:01:13.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.370+0000 7faba17fa640 1 --2- 172.21.15.69:0/3237452239 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fab840744b0 0x7fab840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:13.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.370+0000 7faba17fa640 1 -- 172.21.15.69:0/3237452239 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7fab8c0b7720 con 0x7faba4143440 2024-04-25T11:01:13.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.370+0000 7faba8e46640 1 --2- 172.21.15.69:0/3237452239 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fab840744b0 0x7fab840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:13.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.371+0000 7faba8e46640 1 --2- 172.21.15.69:0/3237452239 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fab840744b0 0x7fab840769c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7faba406dff0 tx=0x7fab94008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:13.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.375+0000 7faba17fa640 1 -- 172.21.15.69:0/3237452239 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fab8c041070 con 0x7faba4143440 2024-04-25T11:01:13.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.548+0000 7fabab0aa640 1 -- 172.21.15.69:0/3237452239 --> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faba40cc7a0 con 0x7fab840744b0 2024-04-25T11:01:13.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.549+0000 7faba17fa640 1 -- 172.21.15.69:0/3237452239 <== mgr.4107 v2:172.21.15.69:6832/1107421267 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13339 (secure 0 0 0) 0x7faba40cc7a0 con 0x7fab840744b0 2024-04-25T11:01:13.549 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:01:13.550 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-04-25T11:01:13.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.551+0000 7fabab0aa640 1 -- 172.21.15.69:0/3237452239 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fab840744b0 msgr2=0x7fab840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:13.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.551+0000 7fabab0aa640 1 --2- 172.21.15.69:0/3237452239 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fab840744b0 0x7fab840769c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7faba406dff0 tx=0x7fab94008040 comp rx=0 tx=0).stop 2024-04-25T11:01:13.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.551+0000 7fabab0aa640 1 -- 172.21.15.69:0/3237452239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faba4143440 msgr2=0x7faba40cdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:13.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.551+0000 7fabab0aa640 1 --2- 172.21.15.69:0/3237452239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faba4143440 0x7faba40cdab0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fab8c000c00 tx=0x7fab8c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:01:13.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.551+0000 7fabab0aa640 1 -- 172.21.15.69:0/3237452239 shutdown_connections 2024-04-25T11:01:13.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.551+0000 7fabab0aa640 1 --2- 172.21.15.69:0/3237452239 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fab840744b0 0x7fab840769c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:13.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.551+0000 7fabab0aa640 1 --2- 172.21.15.69:0/3237452239 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faba41610a0 0x7faba40c7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:13.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.551+0000 7fabab0aa640 1 --2- 172.21.15.69:0/3237452239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faba4143440 0x7faba40cdab0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:13.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.551+0000 7fabab0aa640 1 --2- 172.21.15.69:0/3237452239 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faba40d0830 0x7faba413f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:13.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.551+0000 7fabab0aa640 1 -- 172.21.15.69:0/3237452239 >> 172.21.15.69:0/3237452239 conn(0x7faba40d1ae0 msgr2=0x7faba41427b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:13.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.551+0000 7fabab0aa640 1 -- 172.21.15.69:0/3237452239 shutdown_connections 2024-04-25T11:01:13.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.551+0000 7fabab0aa640 1 -- 172.21.15.69:0/3237452239 wait complete. 2024-04-25T11:01:13.562 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2024-04-25T11:01:12.428811+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220504,"kb_used_data":4328,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754216,"statfs":{"total":773094113280,"available":772868317184,"internally_reserved":0,"allocated":4431872,"data_stored":2448240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002911"},"pg_stats":[{"pgid":"1.0","version":"35'74","reported_seq":149,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:00:53.880100+0000","last_change":"2024-04-25T11:00:53.880100+0000","last_active":"2024-04-25T11:00:53.880100+0000","last_peered":"2024-04-25T11:00:53.880100+0000","last_clean":"2024-04-25T11:00:53.880100+0000","last_became_active":"2024-04-25T10:59:07.836269+0000","last_became_peered":"2024-04-25T10:59:07.836269+0000","last_unstale":"2024-04-25T11:00:53.880100+0000","last_undegraded":"2024-04-25T11:00:53.880100+0000","last_fullsized":"2024-04-25T11:00:53.880100+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'74","last_scrub_stamp":"2024-04-25T11:00:53.879945+0000","last_deep_scrub":"35'74","last_deep_scrub_stamp":"2024-04-25T11:00:53.879945+0000","last_clean_scrub_stamp":"2024-04-25T11:00:53.879945+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:35.633646+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":999424,"data_stored":748806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":158438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":158438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":158438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":158438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":158438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":158438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":999424,"data_stored":748806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:01:13.563 DEBUG:teuthology.orchestra.run.smithi069:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T11:01:13.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.670+0000 7fc56c906640 1 Processor -- start 2024-04-25T11:01:13.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.670+0000 7fc56c906640 1 -- start start 2024-04-25T11:01:13.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.671+0000 7fc56c906640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5680ce1c0 0x7fc5680cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:13.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.671+0000 7fc56c906640 1 -- --> v1:172.21.15.69:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5680ce5d0 con 0x7fc5680d2580 2024-04-25T11:01:13.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.671+0000 7fc56c906640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5680cf3c0 con 0x7fc5680b3f20 2024-04-25T11:01:13.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.671+0000 7fc56c906640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc5680cf6a0 con 0x7fc5680ce1c0 2024-04-25T11:01:13.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.671+0000 7fc565d9b640 1 --1- >> v1:172.21.15.69:6789/0 conn(0x7fc5680d2580 0x7fc5680d2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.69:6789/0 says I am v1:172.21.15.69:47924/0 (socket says 172.21.15.69:47924) 2024-04-25T11:01:13.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.671+0000 7fc565d9b640 1 -- 172.21.15.69:0/3198284695 learned_addr learned my addr 172.21.15.69:0/3198284695 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:01:13.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.672+0000 7fc56559a640 1 -- 172.21.15.69:0/3198284695 <== mon.0 v1:172.21.15.69:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1003124833 0 0) 0x7fc5680ce5d0 con 0x7fc5680d2580 2024-04-25T11:01:13.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.672+0000 7fc56559a640 1 -- 172.21.15.69:0/3198284695 --> v1:172.21.15.69:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc550003520 con 0x7fc5680d2580 2024-04-25T11:01:13.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.672+0000 7fc56659c640 1 --2- 172.21.15.69:0/3198284695 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5680ce1c0 0x7fc5680cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:13.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.672+0000 7fc56559a640 1 -- 172.21.15.69:0/3198284695 <== mon.0 v1:172.21.15.69:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 669613422 0 0) 0x7fc550003520 con 0x7fc5680d2580 2024-04-25T11:01:13.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.672+0000 7fc56559a640 1 -- 172.21.15.69:0/3198284695 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5680ce1c0 msgr2=0x7fc5680cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:13.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.672+0000 7fc56559a640 1 --2- 172.21.15.69:0/3198284695 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5680ce1c0 0x7fc5680cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:13.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.672+0000 7fc56559a640 1 -- 172.21.15.69:0/3198284695 >> v1:172.21.15.146:6789/0 conn(0x7fc5680b3f20 legacy=0x7fc5680b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:13.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.672+0000 7fc56559a640 1 -- 172.21.15.69:0/3198284695 --> v1:172.21.15.69:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5680cf980 con 0x7fc5680d2580 2024-04-25T11:01:13.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.672+0000 7fc56559a640 1 -- 172.21.15.69:0/3198284695 <== mon.0 v1:172.21.15.69:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1972751206 0 0) 0x7fc55c001670 con 0x7fc5680d2580 2024-04-25T11:01:13.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.672+0000 7fc56559a640 1 -- 172.21.15.69:0/3198284695 >> v1:172.21.15.69:6789/0 conn(0x7fc5680d2580 legacy=0x7fc5680d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:13.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.672+0000 7fc56559a640 1 --2- 172.21.15.69:0/3198284695 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5500039e0 0x7fc550003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:13.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.672+0000 7fc56559a640 1 --2- 172.21.15.69:0/3198284695 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5680ce1c0 0x7fc550004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:13.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.672+0000 7fc56559a640 1 --2- 172.21.15.69:0/3198284695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc550004d40 0x7fc550005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:13.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.672+0000 7fc56559a640 1 -- 172.21.15.69:0/3198284695 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc5680cf6a0 con 0x7fc550004d40 2024-04-25T11:01:13.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.672+0000 7fc56559a640 1 -- 172.21.15.69:0/3198284695 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc55000bc10 con 0x7fc5500039e0 2024-04-25T11:01:13.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.672+0000 7fc56559a640 1 -- 172.21.15.69:0/3198284695 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc55000be10 con 0x7fc5680ce1c0 2024-04-25T11:01:13.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.672+0000 7fc566d9d640 1 --2- 172.21.15.69:0/3198284695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc550004d40 0x7fc550005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:13.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.672+0000 7fc56659c640 1 --2- 172.21.15.69:0/3198284695 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5500039e0 0x7fc550003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:13.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.672+0000 7fc566d9d640 1 -- 172.21.15.69:0/3198284695 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5680ce1c0 msgr2=0x7fc550004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:13.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.672+0000 7fc566d9d640 1 --2- 172.21.15.69:0/3198284695 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5680ce1c0 0x7fc550004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:13.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.672+0000 7fc566d9d640 1 -- 172.21.15.69:0/3198284695 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5500039e0 msgr2=0x7fc550003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:13.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.672+0000 7fc566d9d640 1 --2- 172.21.15.69:0/3198284695 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5500039e0 0x7fc550003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:13.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.672+0000 7fc566d9d640 1 -- 172.21.15.69:0/3198284695 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc550003520 con 0x7fc550004d40 2024-04-25T11:01:13.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.672+0000 7fc56659c640 1 --2- 172.21.15.69:0/3198284695 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5500039e0 0x7fc550003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:01:13.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.673+0000 7fc566d9d640 1 --2- 172.21.15.69:0/3198284695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc550004d40 0x7fc550005210 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc568074200 tx=0x7fc550005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606e3db5c973621f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:13.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.673+0000 7fc56559a640 1 -- 172.21.15.69:0/3198284695 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc560037070 con 0x7fc550004d40 2024-04-25T11:01:13.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.673+0000 7fc56559a640 1 -- 172.21.15.69:0/3198284695 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc56002a9d0 con 0x7fc550004d40 2024-04-25T11:01:13.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.673+0000 7fc56559a640 1 -- 172.21.15.69:0/3198284695 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5600039a0 con 0x7fc550004d40 2024-04-25T11:01:13.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.674+0000 7fc56c906640 1 -- 172.21.15.69:0/3198284695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc550004d40 msgr2=0x7fc550005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:13.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.674+0000 7fc56c906640 1 --2- 172.21.15.69:0/3198284695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc550004d40 0x7fc550005210 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc568074200 tx=0x7fc550005830 comp rx=0 tx=0).stop 2024-04-25T11:01:13.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.674+0000 7fc566d9d640 1 -- 172.21.15.69:0/3198284695 reap_dead start 2024-04-25T11:01:13.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.674+0000 7fc56c906640 1 -- 172.21.15.69:0/3198284695 shutdown_connections 2024-04-25T11:01:13.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.674+0000 7fc56c906640 1 -- 172.21.15.69:0/3198284695 >> 172.21.15.69:0/3198284695 conn(0x7fc56809b7d0 msgr2=0x7fc56809bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:13.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.674+0000 7fc56c906640 1 -- 172.21.15.69:0/3198284695 shutdown_connections 2024-04-25T11:01:13.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.674+0000 7fc56c906640 1 -- 172.21.15.69:0/3198284695 wait complete. 2024-04-25T11:01:13.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.676+0000 7fc56c906640 1 Processor -- start 2024-04-25T11:01:13.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.676+0000 7fc56c906640 1 -- start start 2024-04-25T11:01:13.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.677+0000 7fc56c906640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc550004d40 0x7fc56815f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:13.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.677+0000 7fc56c906640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5681622d0 0x7fc56809c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:13.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.677+0000 7fc56c906640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc56809cda0 0x7fc5680c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:13.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.677+0000 7fc56c906640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc5680d3220 con 0x7fc5681622d0 2024-04-25T11:01:13.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.677+0000 7fc56c906640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc5680d30a0 con 0x7fc56809cda0 2024-04-25T11:01:13.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.677+0000 7fc56c906640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7fc5680cec90 con 0x7fc550004d40 2024-04-25T11:01:13.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.677+0000 7fc565d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5681622d0 0x7fc56809c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:13.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.677+0000 7fc565d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5681622d0 0x7fc56809c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41328/0 (socket says 172.21.15.69:41328) 2024-04-25T11:01:13.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.677+0000 7fc565d9b640 1 -- 172.21.15.69:0/3329659770 learned_addr learned my addr 172.21.15.69:0/3329659770 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:01:13.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.678+0000 7fc56659c640 1 --2- 172.21.15.69:0/3329659770 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc550004d40 0x7fc56815f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:13.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.678+0000 7fc566d9d640 1 --2- 172.21.15.69:0/3329659770 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc56809cda0 0x7fc5680c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:13.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.678+0000 7fc565d9b640 1 -- 172.21.15.69:0/3329659770 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc550004d40 msgr2=0x7fc56815f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:13.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.678+0000 7fc565d9b640 1 --2- 172.21.15.69:0/3329659770 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc550004d40 0x7fc56815f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:13.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.678+0000 7fc565d9b640 1 -- 172.21.15.69:0/3329659770 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc56809cda0 msgr2=0x7fc5680c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:13.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.678+0000 7fc565d9b640 1 --2- 172.21.15.69:0/3329659770 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc56809cda0 0x7fc5680c85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:13.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.678+0000 7fc565d9b640 1 -- 172.21.15.69:0/3329659770 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc56809d1b0 con 0x7fc5681622d0 2024-04-25T11:01:13.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.678+0000 7fc56659c640 1 --2- 172.21.15.69:0/3329659770 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc550004d40 0x7fc56815f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:01:13.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.678+0000 7fc566d9d640 1 --2- 172.21.15.69:0/3329659770 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc56809cda0 0x7fc5680c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:01:13.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.678+0000 7fc565d9b640 1 --2- 172.21.15.69:0/3329659770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5681622d0 0x7fc56809c860 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc568163fd0 tx=0x7fc55c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:13.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.678+0000 7fc556ffd640 1 -- 172.21.15.69:0/3329659770 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55c018070 con 0x7fc5681622d0 2024-04-25T11:01:13.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.678+0000 7fc556ffd640 1 -- 172.21.15.69:0/3329659770 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc55c0072f0 con 0x7fc5681622d0 2024-04-25T11:01:13.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.678+0000 7fc56c906640 1 -- 172.21.15.69:0/3329659770 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc568162dc0 con 0x7fc5681622d0 2024-04-25T11:01:13.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.678+0000 7fc556ffd640 1 -- 172.21.15.69:0/3329659770 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55c0047e0 con 0x7fc5681622d0 2024-04-25T11:01:13.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.678+0000 7fc56c906640 1 -- 172.21.15.69:0/3329659770 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5680c7ae0 con 0x7fc5681622d0 2024-04-25T11:01:13.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.679+0000 7fc56c906640 1 -- 172.21.15.69:0/3329659770 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc568162b30 con 0x7fc5681622d0 2024-04-25T11:01:13.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.680+0000 7fc556ffd640 1 -- 172.21.15.69:0/3329659770 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc55c01b070 con 0x7fc5681622d0 2024-04-25T11:01:13.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.680+0000 7fc556ffd640 1 --2- 172.21.15.69:0/3329659770 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc5480744b0 0x7fc5480769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:13.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.680+0000 7fc56659c640 1 --2- 172.21.15.69:0/3329659770 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc5480744b0 0x7fc5480769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:13.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.680+0000 7fc556ffd640 1 -- 172.21.15.69:0/3329659770 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7fc55c0944e0 con 0x7fc5681622d0 2024-04-25T11:01:13.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.681+0000 7fc56659c640 1 --2- 172.21.15.69:0/3329659770 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc5480744b0 0x7fc5480769c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc558009540 tx=0x7fc55800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:13.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.685+0000 7fc556ffd640 1 -- 172.21.15.69:0/3329659770 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc56809d1b0 con 0x7fc5681622d0 2024-04-25T11:01:13.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.864+0000 7fc56c906640 1 -- 172.21.15.69:0/3329659770 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fc568050740 con 0x7fc5681622d0 2024-04-25T11:01:13.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.867+0000 7fc556ffd640 1 -- 172.21.15.69:0/3329659770 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7fc55c05ef10 con 0x7fc5681622d0 2024-04-25T11:01:13.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.868+0000 7fc556ffd640 1 -- 172.21.15.69:0/3329659770 <== mon.0 v2:172.21.15.69:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fc55c05e320 con 0x7fc5681622d0 2024-04-25T11:01:13.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.870+0000 7fc56c906640 1 -- 172.21.15.69:0/3329659770 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc5480744b0 msgr2=0x7fc5480769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:13.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.870+0000 7fc56c906640 1 --2- 172.21.15.69:0/3329659770 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc5480744b0 0x7fc5480769c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc558009540 tx=0x7fc55800b040 comp rx=0 tx=0).stop 2024-04-25T11:01:13.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.870+0000 7fc56c906640 1 -- 172.21.15.69:0/3329659770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5681622d0 msgr2=0x7fc56809c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:13.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.870+0000 7fc56c906640 1 --2- 172.21.15.69:0/3329659770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5681622d0 0x7fc56809c860 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc568163fd0 tx=0x7fc55c00cb90 comp rx=0 tx=0).stop 2024-04-25T11:01:13.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.871+0000 7fc56c906640 1 -- 172.21.15.69:0/3329659770 shutdown_connections 2024-04-25T11:01:13.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.871+0000 7fc56c906640 1 --2- 172.21.15.69:0/3329659770 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7fc5480744b0 0x7fc5480769c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:13.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.871+0000 7fc56c906640 1 --2- 172.21.15.69:0/3329659770 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc56809cda0 0x7fc5680c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:13.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.871+0000 7fc56c906640 1 --2- 172.21.15.69:0/3329659770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5681622d0 0x7fc56809c860 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:13.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.871+0000 7fc56c906640 1 --2- 172.21.15.69:0/3329659770 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc550004d40 0x7fc56815f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:13.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.871+0000 7fc56c906640 1 -- 172.21.15.69:0/3329659770 >> 172.21.15.69:0/3329659770 conn(0x7fc56809b7d0 msgr2=0x7fc5680715e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:13.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.871+0000 7fc56c906640 1 -- 172.21.15.69:0/3329659770 shutdown_connections 2024-04-25T11:01:13.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.871+0000 7fc56c906640 1 -- 172.21.15.69:0/3329659770 wait complete. 2024-04-25T11:01:13.883 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T11:01:13.883 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:01:13.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.989+0000 7f639013f640 1 Processor -- start 2024-04-25T11:01:14.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.989+0000 7f639013f640 1 -- start start 2024-04-25T11:01:14.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.990+0000 7f639013f640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f63880d4570 0x7f63880b4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:14.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.990+0000 7f639013f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63880b2dc0 0x7f63880b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:14.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.990+0000 7f639013f640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f638806d870 con 0x7f638806e8f0 2024-04-25T11:01:14.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.990+0000 7f639013f640 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 -- 0x7f638806e460 con 0x7f63880d4570 2024-04-25T11:01:14.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.990+0000 7f639013f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f638806db10 con 0x7f63880b2dc0 2024-04-25T11:01:14.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.991+0000 7f638e6dc640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63880b2dc0 0x7f63880b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:14.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.991+0000 7f638d6da640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f638806e8f0 0x7f63880d04b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.69:34618/0 (socket says 172.21.15.69:34618) 2024-04-25T11:01:14.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.991+0000 7f638dedb640 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f63880d4570 0x7f63880b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:14.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.991+0000 7f638d6da640 1 -- 172.21.15.69:0/1595121716 learned_addr learned my addr 172.21.15.69:0/1595121716 (peer_addr_for_me v1:172.21.15.69:0/0) 2024-04-25T11:01:14.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.991+0000 7f638e6dc640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63880b2dc0 0x7f63880b31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.69:35424/0 (socket says 172.21.15.69:35424) 2024-04-25T11:01:14.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.991+0000 7f638ced9640 1 -- 172.21.15.69:0/1595121716 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2022595482 0 0) 0x7f638806d870 con 0x7f638806e8f0 2024-04-25T11:01:14.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.991+0000 7f638ced9640 1 -- 172.21.15.69:0/1595121716 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6370003520 con 0x7f638806e8f0 2024-04-25T11:01:14.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.991+0000 7f638dedb640 1 -- 172.21.15.69:0/1595121716 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63880b2dc0 msgr2=0x7f63880b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:01:14.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.991+0000 7f638dedb640 1 --2- 172.21.15.69:0/1595121716 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63880b2dc0 0x7f63880b31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:14.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.991+0000 7f638dedb640 1 -- 172.21.15.69:0/1595121716 >> v1:172.21.15.146:6789/0 conn(0x7f638806e8f0 legacy=0x7f63880d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:14.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.991+0000 7f638dedb640 1 -- 172.21.15.69:0/1595121716 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f638806e0f0 con 0x7f63880d4570 2024-04-25T11:01:14.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.991+0000 7f638e6dc640 1 --2- 172.21.15.69:0/1595121716 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63880b2dc0 0x7f63880b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:01:14.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.992+0000 7f638dedb640 1 --2- 172.21.15.69:0/1595121716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63880d4570 0x7f63880b4830 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6380009790 tx=0x7f638002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3762996877018dd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:14.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.992+0000 7f638ced9640 1 -- 172.21.15.69:0/1595121716 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6380037070 con 0x7f63880d4570 2024-04-25T11:01:14.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.992+0000 7f638ced9640 1 -- 172.21.15.69:0/1595121716 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f638002ae50 con 0x7f63880d4570 2024-04-25T11:01:14.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.992+0000 7f638ced9640 1 -- 172.21.15.69:0/1595121716 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6380032640 con 0x7f63880d4570 2024-04-25T11:01:14.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.993+0000 7f639013f640 1 -- 172.21.15.69:0/1595121716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63880d4570 msgr2=0x7f63880b4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:14.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.993+0000 7f639013f640 1 --2- 172.21.15.69:0/1595121716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63880d4570 0x7f63880b4830 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6380009790 tx=0x7f638002a100 comp rx=0 tx=0).stop 2024-04-25T11:01:14.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.993+0000 7f639013f640 1 -- 172.21.15.69:0/1595121716 shutdown_connections 2024-04-25T11:01:14.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.993+0000 7f639013f640 1 --2- 172.21.15.69:0/1595121716 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63880b2dc0 0x7f63880b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:14.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.993+0000 7f639013f640 1 --2- 172.21.15.69:0/1595121716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63880d4570 0x7f63880b4830 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:14.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.993+0000 7f639013f640 1 -- 172.21.15.69:0/1595121716 >> 172.21.15.69:0/1595121716 conn(0x7f63880d3a50 msgr2=0x7f63880d3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:14.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.993+0000 7f639013f640 1 -- 172.21.15.69:0/1595121716 shutdown_connections 2024-04-25T11:01:14.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.993+0000 7f639013f640 1 -- 172.21.15.69:0/1595121716 wait complete. 2024-04-25T11:01:14.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.995+0000 7f639013f640 1 Processor -- start 2024-04-25T11:01:14.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.995+0000 7f639013f640 1 -- start start 2024-04-25T11:01:14.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.996+0000 7f639013f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f638806e8f0 0x7f63881510e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:14.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.996+0000 7f639013f640 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63880b2dc0 0x7f6388153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:14.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.996+0000 7f639013f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63880d4570 0x7f6388153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:14.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.996+0000 7f639013f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f638806ed00 con 0x7f63880d4570 2024-04-25T11:01:14.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.996+0000 7f639013f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f63880d0fb0 con 0x7f638806e8f0 2024-04-25T11:01:14.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.996+0000 7f639013f640 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 -- 0x7f63880b3710 con 0x7f63880b2dc0 2024-04-25T11:01:14.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.996+0000 7f638e6dc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63880d4570 0x7f6388153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:14.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.996+0000 7f638e6dc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63880d4570 0x7f6388153f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41330/0 (socket says 172.21.15.69:41330) 2024-04-25T11:01:14.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.996+0000 7f638e6dc640 1 -- 172.21.15.69:0/93425596 learned_addr learned my addr 172.21.15.69:0/93425596 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:01:14.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.996+0000 7f638dedb640 1 --2- 172.21.15.69:0/93425596 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f638806e8f0 0x7f63881510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:14.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.996+0000 7f638d6da640 1 --2- 172.21.15.69:0/93425596 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63880b2dc0 0x7f6388153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:14.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.997+0000 7f638e6dc640 1 -- 172.21.15.69:0/93425596 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63880b2dc0 msgr2=0x7f6388153a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:14.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.997+0000 7f638e6dc640 1 --2- 172.21.15.69:0/93425596 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63880b2dc0 0x7f6388153a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:14.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.997+0000 7f638e6dc640 1 -- 172.21.15.69:0/93425596 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f638806e8f0 msgr2=0x7f63881510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:14.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.997+0000 7f638e6dc640 1 --2- 172.21.15.69:0/93425596 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f638806e8f0 0x7f63881510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:14.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.997+0000 7f638e6dc640 1 -- 172.21.15.69:0/93425596 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63880b8370 con 0x7f63880d4570 2024-04-25T11:01:14.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.997+0000 7f638e6dc640 1 --2- 172.21.15.69:0/93425596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63880d4570 0x7f6388153f70 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6378009e60 tx=0x7f637800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:14.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.997+0000 7f63867fc640 1 -- 172.21.15.69:0/93425596 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6378018030 con 0x7f63880d4570 2024-04-25T11:01:14.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.997+0000 7f63867fc640 1 -- 172.21.15.69:0/93425596 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f637800cdf0 con 0x7f63880d4570 2024-04-25T11:01:14.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.997+0000 7f639013f640 1 -- 172.21.15.69:0/93425596 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6388151a40 con 0x7f63880d4570 2024-04-25T11:01:14.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.997+0000 7f63867fc640 1 -- 172.21.15.69:0/93425596 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6378003bd0 con 0x7f63880d4570 2024-04-25T11:01:14.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.997+0000 7f639013f640 1 -- 172.21.15.69:0/93425596 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6388152000 con 0x7f63880d4570 2024-04-25T11:01:14.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.998+0000 7f639013f640 1 -- 172.21.15.69:0/93425596 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6388050740 con 0x7f63880d4570 2024-04-25T11:01:14.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.999+0000 7f63867fc640 1 -- 172.21.15.69:0/93425596 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f637801b070 con 0x7f63880d4570 2024-04-25T11:01:14.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.999+0000 7f63867fc640 1 --2- 172.21.15.69:0/93425596 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f63740744b0 0x7f63740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:01:14.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.999+0000 7f638dedb640 1 --2- 172.21.15.69:0/93425596 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f63740744b0 0x7f63740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:01:14.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:13.999+0000 7f63867fc640 1 -- 172.21.15.69:0/93425596 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5029+0+0 (secure 0 0 0) 0x7f63780959a0 con 0x7f63880d4570 2024-04-25T11:01:14.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:14.000+0000 7f638dedb640 1 --2- 172.21.15.69:0/93425596 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f63740744b0 0x7f63740769c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6380034000 tx=0x7f638002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:01:14.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:14.004+0000 7f63867fc640 1 -- 172.21.15.69:0/93425596 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6378010040 con 0x7f63880d4570 2024-04-25T11:01:14.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:14.215+0000 7f639013f640 1 -- 172.21.15.69:0/93425596 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63880d1220 con 0x7f63880d4570 2024-04-25T11:01:14.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:14.216+0000 7f63867fc640 1 -- 172.21.15.69:0/93425596 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+3866 (secure 0 0 0) 0x7f637805fd00 con 0x7f63880d4570 2024-04-25T11:01:14.217 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:01:14.217 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":23,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2223802951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":2223802951},{"type":"v1","addr":"172.21.15.146:6835","nonce":2223802951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4867,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/8243043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":8243043},{"type":"v1","addr":"172.21.15.146:6834","nonce":8243043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5250,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1639802532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6835","nonce":1639802532},{"type":"v1","addr":"172.21.15.69:6837","nonce":1639802532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5406,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6836/3023804620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3023804620},{"type":"v1","addr":"172.21.15.69:6836","nonce":3023804620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[]} 2024-04-25T11:01:14.218 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 23 2024-04-25T11:01:14.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:14.218+0000 7f639013f640 1 -- 172.21.15.69:0/93425596 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f63740744b0 msgr2=0x7f63740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:14.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:14.218+0000 7f639013f640 1 --2- 172.21.15.69:0/93425596 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f63740744b0 0x7f63740769c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6380034000 tx=0x7f638002e000 comp rx=0 tx=0).stop 2024-04-25T11:01:14.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:14.218+0000 7f639013f640 1 -- 172.21.15.69:0/93425596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63880d4570 msgr2=0x7f6388153f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:01:14.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:14.218+0000 7f639013f640 1 --2- 172.21.15.69:0/93425596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63880d4570 0x7f6388153f70 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6378009e60 tx=0x7f637800c490 comp rx=0 tx=0).stop 2024-04-25T11:01:14.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:14.218+0000 7f639013f640 1 -- 172.21.15.69:0/93425596 shutdown_connections 2024-04-25T11:01:14.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:14.218+0000 7f639013f640 1 --2- 172.21.15.69:0/93425596 >> [v2:172.21.15.69:6832/1107421267,v1:172.21.15.69:6833/1107421267] conn(0x7f63740744b0 0x7f63740769c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:14.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:14.218+0000 7f639013f640 1 --2- 172.21.15.69:0/93425596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63880d4570 0x7f6388153f70 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:14.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:14.218+0000 7f639013f640 1 --2- 172.21.15.69:0/93425596 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63880b2dc0 0x7f6388153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:14.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:14.218+0000 7f639013f640 1 --2- 172.21.15.69:0/93425596 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f638806e8f0 0x7f63881510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:01:14.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:14.218+0000 7f639013f640 1 -- 172.21.15.69:0/93425596 >> 172.21.15.69:0/93425596 conn(0x7f63880d3a50 msgr2=0x7f63880b5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:01:14.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:14.218+0000 7f639013f640 1 -- 172.21.15.69:0/93425596 shutdown_connections 2024-04-25T11:01:14.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:01:14.218+0000 7f639013f640 1 -- 172.21.15.69:0/93425596 wait complete. 2024-04-25T11:01:14.230 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-25T11:01:14.230 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T11:01:14.230 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T11:01:14.230 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:01:14.242 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T11:01:14.242 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T11:01:14.242 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:01:14.294 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T11:01:14.295 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T11:01:14.295 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:01:14.308 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T11:01:14.308 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T11:01:14.308 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:01:14.360 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T11:01:14.361 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T11:01:14.361 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T11:01:14.361 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:01:14.395 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T11:01:14.395 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T11:01:14.395 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:01:14.464 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T11:01:14.465 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T11:01:14.465 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:01:14.542 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T11:01:14.543 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T11:01:14.543 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:01:14.619 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T11:01:14.619 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T11:01:14.619 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:01:14.650 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T11:01:14.651 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T11:01:14.651 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:01:14.718 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T11:01:14.719 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T11:01:14.719 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:01:14.787 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T11:01:14.787 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T11:01:14.787 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:01:14.860 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T11:01:14.861 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T11:01:14.861 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T11:01:14.861 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:01:14.888 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T11:01:14.889 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T11:01:14.889 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:01:14.929 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T11:01:14.929 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T11:01:14.929 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T11:01:14.929 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:01:14.941 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T11:01:14.941 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T11:01:14.941 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:01:14.982 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T11:01:14.982 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T11:01:14.982 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:01:15.035 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T11:01:15.035 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T11:01:15.035 DEBUG:teuthology.orchestra.run.smithi069:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Replacing daemon mds.a' | head -n 1 2024-04-25T11:01:15.079 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi069.front.sepia.ceph.com 2024-04-25T11:01:15.079 DEBUG:teuthology.orchestra.run.smithi069:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T11:01:15.268 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi069.front.sepia.ceph.com 2024-04-25T11:01:15.268 DEBUG:teuthology.orchestra.run.smithi069:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T11:01:15.332 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi069.front.sepia.ceph.com 2024-04-25T11:01:15.332 DEBUG:teuthology.orchestra.run.smithi069:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T11:01:15.390 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi069.front.sepia.ceph.com 2024-04-25T11:01:15.390 DEBUG:teuthology.orchestra.run.smithi069:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T11:01:15.440 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi146.front.sepia.ceph.com 2024-04-25T11:01:15.440 DEBUG:teuthology.orchestra.run.smithi146:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T11:01:15.608 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi146.front.sepia.ceph.com 2024-04-25T11:01:15.609 DEBUG:teuthology.orchestra.run.smithi146:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T11:01:15.689 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi146.front.sepia.ceph.com 2024-04-25T11:01:15.689 DEBUG:teuthology.orchestra.run.smithi146:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T11:01:15.747 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi146.front.sepia.ceph.com 2024-04-25T11:01:15.747 DEBUG:teuthology.orchestra.run.smithi146:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T11:01:15.795 INFO:tasks.ceph:Archiving mon data... 2024-04-25T11:01:15.796 DEBUG:teuthology.misc:Transferring archived files from smithi069:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673109/data/mon.a.tgz 2024-04-25T11:01:15.796 DEBUG:teuthology.orchestra.run.smithi069:> mktemp 2024-04-25T11:01:15.811 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/tmp.ajVZOzXm3u 2024-04-25T11:01:15.812 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.ajVZOzXm3u 2024-04-25T11:01:15.937 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 0666 /tmp/tmp.ajVZOzXm3u 2024-04-25T11:01:16.019 DEBUG:teuthology.orchestra.remote:smithi069:/tmp/tmp.ajVZOzXm3u is 285KB 2024-04-25T11:01:16.114 DEBUG:teuthology.orchestra.run.smithi069:> rm -fr /tmp/tmp.ajVZOzXm3u 2024-04-25T11:01:16.128 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673109/data/mon.b.tgz 2024-04-25T11:01:16.129 DEBUG:teuthology.orchestra.run.smithi146:> mktemp 2024-04-25T11:01:16.147 INFO:teuthology.orchestra.run.smithi146.stdout:/tmp/tmp.lVcRaSlQaH 2024-04-25T11:01:16.148 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.lVcRaSlQaH 2024-04-25T11:01:16.300 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 0666 /tmp/tmp.lVcRaSlQaH 2024-04-25T11:01:16.383 DEBUG:teuthology.orchestra.remote:smithi146:/tmp/tmp.lVcRaSlQaH is 385KB 2024-04-25T11:01:16.480 DEBUG:teuthology.orchestra.run.smithi146:> rm -fr /tmp/tmp.lVcRaSlQaH 2024-04-25T11:01:16.495 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673109/data/mon.c.tgz 2024-04-25T11:01:16.495 DEBUG:teuthology.orchestra.run.smithi146:> mktemp 2024-04-25T11:01:16.549 INFO:teuthology.orchestra.run.smithi146.stdout:/tmp/tmp.j3tomWIxEh 2024-04-25T11:01:16.549 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.j3tomWIxEh 2024-04-25T11:01:16.682 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 0666 /tmp/tmp.j3tomWIxEh 2024-04-25T11:01:16.763 DEBUG:teuthology.orchestra.remote:smithi146:/tmp/tmp.j3tomWIxEh is 385KB 2024-04-25T11:01:16.861 DEBUG:teuthology.orchestra.run.smithi146:> rm -fr /tmp/tmp.j3tomWIxEh 2024-04-25T11:01:16.875 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T11:01:16.875 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T11:01:16.877 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T11:01:16.956 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T11:01:16.994 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T11:01:16.994 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T11:01:16.996 DEBUG:teuthology.misc:Transferring archived files from smithi069:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673109/remote/smithi069/crash 2024-04-25T11:01:16.997 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T11:01:17.021 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673109/remote/smithi146/crash 2024-04-25T11:01:17.022 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T11:01:17.057 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T11:01:17.057 DEBUG:teuthology.orchestra.run.smithi069:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:01:17.066 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:01:17.144 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T11:01:17.168 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T11:01:17.192 INFO:tasks.ceph:Compressing logs... 2024-04-25T11:01:17.192 DEBUG:teuthology.orchestra.run.smithi069:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T11:01:17.211 DEBUG:teuthology.orchestra.run.smithi146:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T11:01:17.281 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T11:01:17.281 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41055.log 2024-04-25T11:01:17.281 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34477.log 2024-04-25T11:01:17.281 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.3.41055.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T11:01:17.281 INFO:teuthology.orchestra.run.smithi146.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41055.log.gz 2024-04-25T11:01:17.281 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34688.log 2024-04-25T11:01:17.281 INFO:teuthology.orchestra.run.smithi146.stderr: 91.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T11:01:17.281 INFO:teuthology.orchestra.run.smithi146.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34477.log.gz 2024-04-25T11:01:17.281 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T11:01:17.282 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34688.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T11:01:17.282 INFO:teuthology.orchestra.run.smithi146.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34688.log.gz 2024-04-25T11:01:17.282 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T11:01:17.282 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34517.log 2024-04-25T11:01:17.282 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T11:01:17.282 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34559.log 2024-04-25T11:01:17.282 INFO:teuthology.orchestra.run.smithi146.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T11:01:17.282 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T11:01:17.282 INFO:teuthology.orchestra.run.smithi146.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34517.log.gz 2024-04-25T11:01:17.282 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34559.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T11:01:17.282 INFO:teuthology.orchestra.run.smithi146.stderr: 87.5%/var/log/ceph/ceph-mgr.x.log: -- replaced with /var/log/ceph/ceph-client.admin.34559.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.1.40658.log 2024-04-25T11:01:17.282 INFO:teuthology.orchestra.run.smithi146.stderr: 2024-04-25T11:01:17.283 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T11:01:17.283 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.1.40658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34608.log 2024-04-25T11:01:17.283 INFO:teuthology.orchestra.run.smithi146.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.40658.log.gz 2024-04-25T11:01:17.283 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34648.log 2024-04-25T11:01:17.283 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34608.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34608.log.gz 2024-04-25T11:01:17.283 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34728.log 2024-04-25T11:01:17.283 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34648.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34648.log.gz 2024-04-25T11:01:17.283 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T11:01:17.283 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34728.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34728.log.gz 2024-04-25T11:01:17.283 INFO:teuthology.orchestra.run.smithi146.stderr: 92.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T11:01:17.283 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T11:01:17.283 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-mds.d.log: 91.8% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T11:01:17.284 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34768.log 2024-04-25T11:01:17.284 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.34768.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34768.log.gz 2024-04-25T11:01:17.312 INFO:teuthology.orchestra.run.smithi146.stderr: 94.4% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T11:01:17.336 INFO:teuthology.orchestra.run.smithi146.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T11:01:17.349 INFO:teuthology.orchestra.run.smithi146.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T11:01:17.353 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45905.log 2024-04-25T11:01:17.353 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48393.log 2024-04-25T11:01:17.353 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.45905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44396.log 2024-04-25T11:01:17.353 INFO:teuthology.orchestra.run.smithi069.stderr: 88.1%/var/log/ceph/ceph-client.admin.48393.log: -- replaced with /var/log/ceph/ceph-client.admin.45905.log.gz 2024-04-25T11:01:17.353 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47871.log 2024-04-25T11:01:17.353 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.44396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44633.log 2024-04-25T11:01:17.353 INFO:teuthology.orchestra.run.smithi069.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48393.log.gz 2024-04-25T11:01:17.353 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.47871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48697.log 2024-04-25T11:01:17.353 INFO:teuthology.orchestra.run.smithi069.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44396.log.gz 2024-04-25T11:01:17.353 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.44633.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47871.log.gz 2024-04-25T11:01:17.354 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40911.log 2024-04-25T11:01:17.354 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.48697.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44633.log.gz 2024-04-25T11:01:17.354 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48823.log 2024-04-25T11:01:17.354 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.40911.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48697.log.gz 2024-04-25T11:01:17.354 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46542.log 2024-04-25T11:01:17.354 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.48823.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40911.log.gz 2024-04-25T11:01:17.354 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47357.log 2024-04-25T11:01:17.354 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.46542.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48823.log.gz 2024-04-25T11:01:17.354 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41296.log 2024-04-25T11:01:17.354 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.47357.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46542.log.gz 2024-04-25T11:01:17.354 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49220.log 2024-04-25T11:01:17.354 INFO:teuthology.orchestra.run.smithi069.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47357.log.gz 2024-04-25T11:01:17.355 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.41296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46416.log 2024-04-25T11:01:17.355 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.49220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42603.log 2024-04-25T11:01:17.355 INFO:teuthology.orchestra.run.smithi069.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41296.log.gz 2024-04-25T11:01:17.355 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.46416.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49220.log.gz 2024-04-25T11:01:17.355 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40995.log 2024-04-25T11:01:17.355 INFO:teuthology.orchestra.run.smithi069.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46416.log.gz 2024-04-25T11:01:17.355 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.42603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44270.log 2024-04-25T11:01:17.355 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.40995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42087.log 2024-04-25T11:01:17.355 INFO:teuthology.orchestra.run.smithi069.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42603.log.gz 2024-04-25T11:01:17.355 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.44270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42213.log 2024-04-25T11:01:17.355 INFO:teuthology.orchestra.run.smithi069.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40995.log.gz 2024-04-25T11:01:17.355 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.42087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46032.log 2024-04-25T11:01:17.355 INFO:teuthology.orchestra.run.smithi069.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44270.log.gz 2024-04-25T11:01:17.356 INFO:teuthology.orchestra.run.smithi069.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42087.log.gz/var/log/ceph/ceph-client.admin.42213.log: 2024-04-25T11:01:17.356 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47695.log 2024-04-25T11:01:17.356 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.46032.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42213.log.gz 2024-04-25T11:01:17.356 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44939.log 2024-04-25T11:01:17.356 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.0.47695.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46032.log.gz 2024-04-25T11:01:17.356 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41079.log 2024-04-25T11:01:17.356 INFO:teuthology.orchestra.run.smithi069.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47695.log.gz 2024-04-25T11:01:17.356 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.44939.log: /var/log/ceph/ceph-client.admin.46084.log 2024-04-25T11:01:17.356 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.41079.log: 89.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.44939.log.gz 2024-04-25T11:01:17.357 INFO:teuthology.orchestra.run.smithi069.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45821.log 2024-04-25T11:01:17.357 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.46084.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41079.log.gz 2024-04-25T11:01:17.357 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43556.log 2024-04-25T11:01:17.357 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.45821.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46084.log.gz 2024-04-25T11:01:17.357 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41961.log 2024-04-25T11:01:17.357 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.43556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50142.log 2024-04-25T11:01:17.357 INFO:teuthology.orchestra.run.smithi069.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45821.log.gz 2024-04-25T11:01:17.357 INFO:teuthology.orchestra.run.smithi069.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43556.log.gz 2024-04-25T11:01:17.357 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.41961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42465.log 2024-04-25T11:01:17.358 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.50142.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41961.log.gz 2024-04-25T11:01:17.358 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44144.log 2024-04-25T11:01:17.358 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.42465.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50142.log.gz 2024-04-25T11:01:17.358 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T11:01:17.358 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.44144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45113.log 2024-04-25T11:01:17.358 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44144.log.gz 2024-04-25T11:01:17.358 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45863.log 2024-04-25T11:01:17.358 INFO:teuthology.orchestra.run.smithi069.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.42465.log.gz 2024-04-25T11:01:17.359 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.45113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41302.log 2024-04-25T11:01:17.359 INFO:teuthology.orchestra.run.smithi069.stderr: 84.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T11:01:17.359 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.45863.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45113.log.gz 2024-04-25T11:01:17.359 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41121.log 2024-04-25T11:01:17.359 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.41302.log: 89.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.45863.log.gz -5 2024-04-25T11:01:17.359 INFO:teuthology.orchestra.run.smithi069.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44981.log 2024-04-25T11:01:17.359 INFO:teuthology.orchestra.run.smithi069.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41302.log.gz 2024-04-25T11:01:17.359 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.41121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48613.log 2024-04-25T11:01:17.360 INFO:teuthology.orchestra.run.smithi069.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41121.log.gz 2024-04-25T11:01:17.360 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48173.log 2024-04-25T11:01:17.360 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.44981.log: /var/log/ceph/ceph-client.admin.48613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45653.log 2024-04-25T11:01:17.360 INFO:teuthology.orchestra.run.smithi069.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44981.log.gz 2024-04-25T11:01:17.360 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.48173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45359.log 2024-04-25T11:01:17.360 INFO:teuthology.orchestra.run.smithi069.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48613.log.gz 2024-04-25T11:01:17.360 INFO:teuthology.orchestra.run.smithi069.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48173.log.gz 2024-04-25T11:01:17.360 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.45653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41635.log 2024-04-25T11:01:17.360 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.45359.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45653.log.gz 2024-04-25T11:01:17.361 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49806.log 2024-04-25T11:01:17.361 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.41635.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45359.log.gz 2024-04-25T11:01:17.361 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T11:01:17.361 INFO:teuthology.orchestra.run.smithi069.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41635.log.gz 2024-04-25T11:01:17.361 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.49806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46668.log 2024-04-25T11:01:17.361 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-osd.2.log: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.49806.log.gz 2024-04-25T11:01:17.361 INFO:teuthology.orchestra.run.smithi069.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41298.log 2024-04-25T11:01:17.361 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.46668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47971.log 2024-04-25T11:01:17.362 INFO:teuthology.orchestra.run.smithi069.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46668.log.gz 2024-04-25T11:01:17.362 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.41298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.43380.log 2024-04-25T11:01:17.362 INFO:teuthology.orchestra.run.smithi069.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41298.log.gz 2024-04-25T11:01:17.362 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.47971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41300.log 2024-04-25T11:01:17.362 INFO:teuthology.orchestra.run.smithi069.stderr: 87.6%/var/log/ceph/ceph-client.2.43380.log: -- replaced with /var/log/ceph/ceph-client.admin.47971.log.gz 2024-04-25T11:01:17.362 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44102.log 2024-04-25T11:01:17.362 INFO:teuthology.orchestra.run.smithi069.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.43380.log.gz 2024-04-25T11:01:17.362 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.41300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46500.log 2024-04-25T11:01:17.362 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.44102.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41300.log.gz 2024-04-25T11:01:17.363 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48435.log 2024-04-25T11:01:17.363 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.46500.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44102.log.gz 2024-04-25T11:01:17.363 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50379.log 2024-04-25T11:01:17.363 INFO:teuthology.orchestra.run.smithi069.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46500.log.gz 2024-04-25T11:01:17.363 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.48435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49848.log 2024-04-25T11:01:17.363 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.50379.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48435.log.gz 2024-04-25T11:01:17.363 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45569.log 2024-04-25T11:01:17.363 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.49848.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50379.log.gz 2024-04-25T11:01:17.364 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47507.log 2024-04-25T11:01:17.364 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.45569.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49848.log.gz 2024-04-25T11:01:17.364 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T11:01:17.364 INFO:teuthology.orchestra.run.smithi069.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45569.log.gz 2024-04-25T11:01:17.364 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.47507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44354.log 2024-04-25T11:01:17.364 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41630.log 2024-04-25T11:01:17.364 INFO:teuthology.orchestra.run.smithi069.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47507.log.gz 2024-04-25T11:01:17.364 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.44354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49890.log 2024-04-25T11:01:17.364 INFO:teuthology.orchestra.run.smithi069.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44354.log.gz 2024-04-25T11:01:17.364 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.41630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40953.log 2024-04-25T11:01:17.365 INFO:teuthology.orchestra.run.smithi069.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41630.log.gz 2024-04-25T11:01:17.365 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48519.log 2024-04-25T11:01:17.365 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.49890.log: /var/log/ceph/ceph-client.admin.40953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45155.log 2024-04-25T11:01:17.365 INFO:teuthology.orchestra.run.smithi069.stderr: 89.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T11:01:17.365 INFO:teuthology.orchestra.run.smithi069.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49890.log.gz 2024-04-25T11:01:17.365 INFO:teuthology.orchestra.run.smithi069.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40953.log.gz 2024-04-25T11:01:17.365 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.48519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49722.log 2024-04-25T11:01:17.365 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.45155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50268.log 2024-04-25T11:01:17.365 INFO:teuthology.orchestra.run.smithi069.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48519.log.gz 2024-04-25T11:01:17.366 INFO:teuthology.orchestra.run.smithi069.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45155.log.gz 2024-04-25T11:01:17.366 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.49722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41634.log 2024-04-25T11:01:17.366 INFO:teuthology.orchestra.run.smithi069.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49722.log.gz 2024-04-25T11:01:17.366 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.50268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47021.log 2024-04-25T11:01:17.366 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.41634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40659.log 2024-04-25T11:01:17.366 INFO:teuthology.orchestra.run.smithi069.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50268.log.gz 2024-04-25T11:01:17.366 INFO:teuthology.orchestra.run.smithi069.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41634.log.gz 2024-04-25T11:01:17.366 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.47021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43598.log 2024-04-25T11:01:17.366 INFO:teuthology.orchestra.run.smithi069.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47021.log.gz 2024-04-25T11:01:17.366 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.40659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46208.log 2024-04-25T11:01:17.367 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.43598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48865.log 2024-04-25T11:01:17.367 INFO:teuthology.orchestra.run.smithi069.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40659.log.gz 2024-04-25T11:01:17.367 INFO:teuthology.orchestra.run.smithi069.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43598.log.gz 2024-04-25T11:01:17.367 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.46208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47809.log 2024-04-25T11:01:17.367 INFO:teuthology.orchestra.run.smithi069.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46208.log.gz 2024-04-25T11:01:17.367 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.48865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41835.log 2024-04-25T11:01:17.367 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.47809.log: gzip 88.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.48865.log.gz --verbose 2024-04-25T11:01:17.367 INFO:teuthology.orchestra.run.smithi069.stderr: -- /var/log/ceph/ceph-client.admin.41636.log 2024-04-25T11:01:17.367 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.41835.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47809.log.gz 2024-04-25T11:01:17.368 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41632.log 2024-04-25T11:01:17.368 INFO:teuthology.orchestra.run.smithi069.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41835.log.gz 2024-04-25T11:01:17.368 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.41636.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T11:01:17.368 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.41632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44438.log 2024-04-25T11:01:17.368 INFO:teuthology.orchestra.run.smithi069.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41636.log.gz 2024-04-25T11:01:17.368 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-osd.3.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41632.log.gz 2024-04-25T11:01:17.368 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49680.log 2024-04-25T11:01:17.368 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.44438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49494.log 2024-04-25T11:01:17.368 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.49680.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44438.log.gz 2024-04-25T11:01:17.368 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48655.log 2024-04-25T11:01:17.368 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49136.log 2024-04-25T11:01:17.369 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.49494.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49680.log.gz 2024-04-25T11:01:17.369 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.48655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45443.log 2024-04-25T11:01:17.369 INFO:teuthology.orchestra.run.smithi069.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49494.log.gz 2024-04-25T11:01:17.369 INFO:teuthology.orchestra.run.smithi069.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48655.log.gz 2024-04-25T11:01:17.369 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.49136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48949.log 2024-04-25T11:01:17.369 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.45443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50310.log 2024-04-25T11:01:17.369 INFO:teuthology.orchestra.run.smithi069.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49136.log.gz 2024-04-25T11:01:17.369 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.48949.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45443.log.gz 2024-04-25T11:01:17.369 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34649.log 2024-04-25T11:01:17.369 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.50310.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48949.log.gz 2024-04-25T11:01:17.369 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48739.log 2024-04-25T11:01:17.369 INFO:teuthology.orchestra.run.smithi069.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50310.log.gz 2024-04-25T11:01:17.369 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.34649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49536.log 2024-04-25T11:01:17.370 INFO:teuthology.orchestra.run.smithi069.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34649.log.gz 2024-04-25T11:01:17.370 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T11:01:17.370 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.48739.log: /var/log/ceph/ceph-client.admin.49536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40219.log 2024-04-25T11:01:17.370 INFO:teuthology.orchestra.run.smithi069.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48739.log.gz 2024-04-25T11:01:17.370 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-mds.c.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49536.log.gz 2024-04-25T11:01:17.370 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46289.log 2024-04-25T11:01:17.370 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49638.log 2024-04-25T11:01:17.370 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.40219.log: /var/log/ceph/ceph-client.admin.46289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46331.log 2024-04-25T11:01:17.370 INFO:teuthology.orchestra.run.smithi069.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40219.log.gz 2024-04-25T11:01:17.370 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.49638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46458.log 2024-04-25T11:01:17.370 INFO:teuthology.orchestra.run.smithi069.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46289.log.gz 2024-04-25T11:01:17.370 INFO:teuthology.orchestra.run.smithi069.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49638.log.gz 2024-04-25T11:01:17.371 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.46331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50184.log 2024-04-25T11:01:17.371 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40300.log 2024-04-25T11:01:17.371 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.46458.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46331.log.gz 2024-04-25T11:01:17.371 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.50184.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T11:01:17.371 INFO:teuthology.orchestra.run.smithi069.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46458.log.gz 2024-04-25T11:01:17.371 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.40300.log: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.50184.log.gz -5 2024-04-25T11:01:17.371 INFO:teuthology.orchestra.run.smithi069.stderr: --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T11:01:17.371 INFO:teuthology.orchestra.run.smithi069.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40300.log.gz 2024-04-25T11:01:17.371 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49452.log 2024-04-25T11:01:17.371 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40869.log 2024-04-25T11:01:17.371 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.49452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44675.log 2024-04-25T11:01:17.371 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.40869.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49452.log.gz 2024-04-25T11:01:17.372 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41877.log 2024-04-25T11:01:17.372 INFO:teuthology.orchestra.run.smithi069.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40869.log.gz 2024-04-25T11:01:17.372 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48024.log 2024-04-25T11:01:17.372 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.44675.log: /var/log/ceph/ceph-client.admin.41877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41294.log 2024-04-25T11:01:17.372 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.48024.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44675.log.gz 2024-04-25T11:01:17.372 INFO:teuthology.orchestra.run.smithi069.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41877.log.gz 2024-04-25T11:01:17.372 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48215.log 2024-04-25T11:01:17.372 INFO:teuthology.orchestra.run.smithi069.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48024.log.gz 2024-04-25T11:01:17.372 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.41294.log: /var/log/ceph/ceph-osd.0.log 2024-04-25T11:01:17.372 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.48215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43514.log 2024-04-25T11:01:17.372 INFO:teuthology.orchestra.run.smithi069.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41294.log.gz 2024-04-25T11:01:17.372 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-osd.0.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.48215.log.gz 2024-04-25T11:01:17.372 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44717.log 2024-04-25T11:01:17.373 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.43514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44837.log 2024-04-25T11:01:17.373 INFO:teuthology.orchestra.run.smithi069.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43514.log.gz 2024-04-25T11:01:17.373 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.44717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48991.log 2024-04-25T11:01:17.373 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.44837.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44717.log.gz 2024-04-25T11:01:17.373 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40342.log 2024-04-25T11:01:17.373 INFO:teuthology.orchestra.run.smithi069.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44837.log.gz 2024-04-25T11:01:17.373 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.48991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44060.log 2024-04-25T11:01:17.373 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.40342.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48991.log.gz 2024-04-25T11:01:17.373 INFO:teuthology.orchestra.run.smithi069.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40342.log.gz 2024-04-25T11:01:17.373 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46979.log 2024-04-25T11:01:17.373 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45779.log 2024-04-25T11:01:17.373 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.44060.log: /var/log/ceph/ceph-client.admin.46979.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44060.log.gz 2024-04-25T11:01:17.373 INFO:teuthology.orchestra.run.smithi069.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46979.log.gz 2024-04-25T11:01:17.374 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48351.log 2024-04-25T11:01:17.374 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40469.log 2024-04-25T11:01:17.374 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.48351.log: /var/log/ceph/ceph-client.admin.45779.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45779.log.gz 2024-04-25T11:01:17.374 INFO:teuthology.orchestra.run.smithi069.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48351.log.gz 2024-04-25T11:01:17.374 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47315.log 2024-04-25T11:01:17.374 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47399.log 2024-04-25T11:01:17.374 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.40469.log: /var/log/ceph/ceph-client.admin.47315.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40469.log.gz 2024-04-25T11:01:17.374 INFO:teuthology.orchestra.run.smithi069.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47315.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49932.log 2024-04-25T11:01:17.374 INFO:teuthology.orchestra.run.smithi069.stderr: 2024-04-25T11:01:17.374 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48085.log 2024-04-25T11:01:17.374 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.47399.log: /var/log/ceph/ceph-client.admin.49932.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47399.log.gz 2024-04-25T11:01:17.374 INFO:teuthology.orchestra.run.smithi069.stderr: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41291.log 2024-04-25T11:01:17.375 INFO:teuthology.orchestra.run.smithi069.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49932.log.gz 2024-04-25T11:01:17.375 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48907.log 2024-04-25T11:01:17.375 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.48085.log: /var/log/ceph/ceph-client.admin.41291.log: 88.4% 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48085.log.gz 2024-04-25T11:01:17.375 INFO:teuthology.orchestra.run.smithi069.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41291.log.gz 2024-04-25T11:01:17.375 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42941.log 2024-04-25T11:01:17.375 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45611.log 2024-04-25T11:01:17.375 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.48907.log: /var/log/ceph/ceph-client.0.42941.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42941.log.gz 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48907.log.gz 2024-04-25T11:01:17.375 INFO:teuthology.orchestra.run.smithi069.stderr: 2024-04-25T11:01:17.375 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45737.log 2024-04-25T11:01:17.375 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41627.log 2024-04-25T11:01:17.375 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.45611.log: /var/log/ceph/ceph-client.admin.45737.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45611.log.gz 2024-04-25T11:01:17.375 INFO:teuthology.orchestra.run.smithi069.stderr: 90.3% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T11:01:17.376 INFO:teuthology.orchestra.run.smithi069.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45737.log.gz 2024-04-25T11:01:17.376 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43892.log 2024-04-25T11:01:17.376 INFO:teuthology.orchestra.run.smithi069.stderr:gzip/var/log/ceph/ceph-client.admin.41627.log: -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T11:01:17.376 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.43892.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41627.log.gz 2024-04-25T11:01:17.376 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46373.log 2024-04-25T11:01:17.376 INFO:teuthology.orchestra.run.smithi069.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43892.log.gz 2024-04-25T11:01:17.376 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50100.log 2024-04-25T11:01:17.376 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.46373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45281.log 2024-04-25T11:01:17.376 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.50100.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46373.log.gz 2024-04-25T11:01:17.376 INFO:teuthology.orchestra.run.smithi069.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50100.log.gz 2024-04-25T11:01:17.376 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50419.log 2024-04-25T11:01:17.376 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.45281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42423.log 2024-04-25T11:01:17.376 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.50419.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45281.log.gz 2024-04-25T11:01:17.377 INFO:teuthology.orchestra.run.smithi069.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50419.log.gz 2024-04-25T11:01:17.377 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43076.log 2024-04-25T11:01:17.377 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.42423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49974.log 2024-04-25T11:01:17.377 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.43076.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42423.log.gz 2024-04-25T11:01:17.377 INFO:teuthology.orchestra.run.smithi069.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43076.log.gz 2024-04-25T11:01:17.377 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40785.log 2024-04-25T11:01:17.377 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.49974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41919.log 2024-04-25T11:01:17.377 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.40785.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49974.log.gz 2024-04-25T11:01:17.377 INFO:teuthology.orchestra.run.smithi069.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40785.log.gz 2024-04-25T11:01:17.377 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42339.log 2024-04-25T11:01:17.377 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.41919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47189.log 2024-04-25T11:01:17.377 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.42339.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41919.log.gz 2024-04-25T11:01:17.377 INFO:teuthology.orchestra.run.smithi069.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42339.log.gz 2024-04-25T11:01:17.378 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42381.log 2024-04-25T11:01:17.378 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.47189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42171.log 2024-04-25T11:01:17.378 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.42381.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47189.log.gz 2024-04-25T11:01:17.378 INFO:teuthology.orchestra.run.smithi069.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42381.log.gzgzip 2024-04-25T11:01:17.378 INFO:teuthology.orchestra.run.smithi069.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.40553.log 2024-04-25T11:01:17.378 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.42171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40385.log 2024-04-25T11:01:17.378 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.40553.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42171.log.gz 2024-04-25T11:01:17.378 INFO:teuthology.orchestra.run.smithi069.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40553.log.gz 2024-04-25T11:01:17.378 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43850.log 2024-04-25T11:01:17.378 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.40385.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40385.log.gz 2024-04-25T11:01:17.378 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46754.log 2024-04-25T11:01:17.378 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.43850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34913.log 2024-04-25T11:01:17.379 INFO:teuthology.orchestra.run.smithi069.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43850.log.gz 2024-04-25T11:01:17.379 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.46754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46584.log 2024-04-25T11:01:17.379 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.34913.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46754.log.gz 2024-04-25T11:01:17.379 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46849.log 2024-04-25T11:01:17.379 INFO:teuthology.orchestra.run.smithi069.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.34913.log.gz 2024-04-25T11:01:17.379 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.46584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41037.log 2024-04-25T11:01:17.379 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.46849.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46584.log.gz 2024-04-25T11:01:17.379 INFO:teuthology.orchestra.run.smithi069.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46849.log.gz 2024-04-25T11:01:17.379 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45485.log 2024-04-25T11:01:17.379 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43976.log 2024-04-25T11:01:17.379 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.45485.log: /var/log/ceph/ceph-client.admin.41037.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45485.log.gz 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41037.log.gz 2024-04-25T11:01:17.379 INFO:teuthology.orchestra.run.smithi069.stderr: 2024-04-25T11:01:17.379 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44759.log 2024-04-25T11:01:17.380 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44018.log 2024-04-25T11:01:17.380 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.43976.log: /var/log/ceph/ceph-client.admin.44759.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43976.log.gz 2024-04-25T11:01:17.380 INFO:teuthology.orchestra.run.smithi069.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44759.log.gz 2024-04-25T11:01:17.380 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42129.log 2024-04-25T11:01:17.380 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44186.log 2024-04-25T11:01:17.380 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.42129.log: /var/log/ceph/ceph-client.admin.44018.log: 88.4% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44018.log.gz 2024-04-25T11:01:17.380 INFO:teuthology.orchestra.run.smithi069.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42129.log.gz 2024-04-25T11:01:17.380 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41639.log 2024-04-25T11:01:17.380 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49033.log 2024-04-25T11:01:17.380 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.44186.log: /var/log/ceph/ceph-client.admin.41639.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44186.log.gz 2024-04-25T11:01:17.380 INFO:teuthology.orchestra.run.smithi069.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41639.log.gz 2024-04-25T11:01:17.380 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40597.log 2024-04-25T11:01:17.380 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49075.log 2024-04-25T11:01:17.381 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.40597.log: /var/log/ceph/ceph-client.admin.49033.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49033.log.gz 2024-04-25T11:01:17.381 INFO:teuthology.orchestra.run.smithi069.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40597.log.gz 2024-04-25T11:01:17.381 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45197.log 2024-04-25T11:01:17.381 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35016.log 2024-04-25T11:01:17.381 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.49075.log: /var/log/ceph/ceph-client.admin.45197.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49075.log.gz 2024-04-25T11:01:17.381 INFO:teuthology.orchestra.run.smithi069.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45527.log 2024-04-25T11:01:17.381 INFO:teuthology.orchestra.run.smithi069.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45197.log.gz 2024-04-25T11:01:17.381 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47147.log 2024-04-25T11:01:17.381 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.35016.log: /var/log/ceph/ceph-client.admin.45527.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.35016.log.gz 2024-04-25T11:01:17.381 INFO:teuthology.orchestra.run.smithi069.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45527.log.gz 2024-04-25T11:01:17.381 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47273.log 2024-04-25T11:01:17.381 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44897.log 2024-04-25T11:01:17.382 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.47147.log: /var/log/ceph/ceph-client.admin.47273.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47147.log.gz 2024-04-25T11:01:17.382 INFO:teuthology.orchestra.run.smithi069.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47273.log.gz 2024-04-25T11:01:17.382 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44591.log 2024-04-25T11:01:17.382 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41303.log 2024-04-25T11:01:17.382 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.44591.log: /var/log/ceph/ceph-client.admin.44897.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44897.log.gz 2024-04-25T11:01:17.382 INFO:teuthology.orchestra.run.smithi069.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44591.log.gz 2024-04-25T11:01:17.382 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50226.log 2024-04-25T11:01:17.382 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42297.log 2024-04-25T11:01:17.382 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.41303.log: /var/log/ceph/ceph-client.admin.50226.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41303.log.gz 2024-04-25T11:01:17.382 INFO:teuthology.orchestra.run.smithi069.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50226.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50461.log 2024-04-25T11:01:17.382 INFO:teuthology.orchestra.run.smithi069.stderr: 2024-04-25T11:01:17.382 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46127.log 2024-04-25T11:01:17.383 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.42297.log: /var/log/ceph/ceph-client.admin.50461.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50461.log.gz 89.2% 2024-04-25T11:01:17.383 INFO:teuthology.orchestra.run.smithi069.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42297.log.gz 2024-04-25T11:01:17.383 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47063.log 2024-04-25T11:01:17.383 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45947.log 2024-04-25T11:01:17.383 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.46127.log: /var/log/ceph/ceph-client.admin.47063.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46127.log.gz 2024-04-25T11:01:17.383 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45023.log 2024-04-25T11:01:17.383 INFO:teuthology.orchestra.run.smithi069.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47063.log.gz 2024-04-25T11:01:17.383 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.45947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48127.log 2024-04-25T11:01:17.383 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.45023.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45947.log.gz 2024-04-25T11:01:17.383 INFO:teuthology.orchestra.run.smithi069.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45023.log.gz 2024-04-25T11:01:17.383 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49262.log 2024-04-25T11:01:17.383 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46798.log 2024-04-25T11:01:17.383 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.48127.log: /var/log/ceph/ceph-client.admin.49262.log: 91.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48127.log.gz 2024-04-25T11:01:17.384 INFO:teuthology.orchestra.run.smithi069.stderr: 2024-04-25T11:01:17.384 INFO:teuthology.orchestra.run.smithi069.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49262.log.gz 2024-04-25T11:01:17.384 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40827.log 2024-04-25T11:01:17.384 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43934.log 2024-04-25T11:01:17.384 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.46798.log: /var/log/ceph/ceph-client.admin.40827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47105.log 2024-04-25T11:01:17.384 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.43934.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46798.log.gz 2024-04-25T11:01:17.384 INFO:teuthology.orchestra.run.smithi069.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40827.log.gz 2024-04-25T11:01:17.384 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45989.log 2024-04-25T11:01:17.384 INFO:teuthology.orchestra.run.smithi069.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.43934.log.gz 2024-04-25T11:01:17.384 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43724.log 2024-04-25T11:01:17.384 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.47105.log: /var/log/ceph/ceph-client.admin.45989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44228.log 2024-04-25T11:01:17.384 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.43724.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47105.log.gz 2024-04-25T11:01:17.384 INFO:teuthology.orchestra.run.smithi069.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45989.log.gz 2024-04-25T11:01:17.385 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45401.log 2024-04-25T11:01:17.385 INFO:teuthology.orchestra.run.smithi069.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43724.log.gz 2024-04-25T11:01:17.385 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44480.log 2024-04-25T11:01:17.385 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.44228.log: /var/log/ceph/ceph-client.admin.45401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50058.log 2024-04-25T11:01:17.385 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.44480.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44228.log.gz 2024-04-25T11:01:17.385 INFO:teuthology.orchestra.run.smithi069.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45401.log.gz 2024-04-25T11:01:17.385 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43766.log 2024-04-25T11:01:17.385 INFO:teuthology.orchestra.run.smithi069.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44480.log.gz 2024-04-25T11:01:17.385 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40701.log 2024-04-25T11:01:17.385 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.50058.log: /var/log/ceph/ceph-client.admin.43766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40123.log 2024-04-25T11:01:17.385 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.40701.log: 88.2% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43766.log.gz -- replaced with /var/log/ceph/ceph-client.admin.50058.log.gz 2024-04-25T11:01:17.385 INFO:teuthology.orchestra.run.smithi069.stderr: 2024-04-25T11:01:17.385 INFO:teuthology.orchestra.run.smithi069.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40701.log.gz 2024-04-25T11:01:17.385 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41292.log 2024-04-25T11:01:17.386 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40427.log 2024-04-25T11:01:17.386 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.40123.log: /var/log/ceph/ceph-client.admin.41292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47441.log 2024-04-25T11:01:17.386 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.40427.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40123.log.gz 2024-04-25T11:01:17.386 INFO:teuthology.orchestra.run.smithi069.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41292.log.gz 2024-04-25T11:01:17.386 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40511.log 2024-04-25T11:01:17.386 INFO:teuthology.orchestra.run.smithi069.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40427.log.gz 2024-04-25T11:01:17.386 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T11:01:17.386 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.47441.log: /var/log/ceph/ceph-client.admin.40511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40743.log 2024-04-25T11:01:17.386 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-mds.a.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40511.log.gz 2024-04-25T11:01:17.386 INFO:teuthology.orchestra.run.smithi069.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.47441.log.gz 2024-04-25T11:01:17.386 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47231.log 2024-04-25T11:01:17.386 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42255.log 2024-04-25T11:01:17.387 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.40743.log: /var/log/ceph/ceph-client.admin.47231.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40743.log.gz 2024-04-25T11:01:17.387 INFO:teuthology.orchestra.run.smithi069.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47231.log.gz 2024-04-25T11:01:17.387 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46710.log 2024-04-25T11:01:17.387 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.42255.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42255.log.gz 2024-04-25T11:01:17.387 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45695.log 2024-04-25T11:01:17.387 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42045.log 2024-04-25T11:01:17.387 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.46710.log: /var/log/ceph/ceph-client.admin.45695.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46710.log.gz 2024-04-25T11:01:17.387 INFO:teuthology.orchestra.run.smithi069.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45695.log.gz 2024-04-25T11:01:17.387 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46891.log 2024-04-25T11:01:17.387 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.42045.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42045.log.gz 2024-04-25T11:01:17.387 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41631.log 2024-04-25T11:01:17.387 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42003.log 2024-04-25T11:01:17.387 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.46891.log: /var/log/ceph/ceph-client.admin.41631.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46891.log.gz 2024-04-25T11:01:17.388 INFO:teuthology.orchestra.run.smithi069.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41631.log.gz 2024-04-25T11:01:17.388 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43808.log 2024-04-25T11:01:17.388 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.42003.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42003.log.gz 2024-04-25T11:01:17.388 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49578.log 2024-04-25T11:01:17.388 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40030.log 2024-04-25T11:01:17.388 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.43808.log: /var/log/ceph/ceph-client.admin.49578.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43808.log.gz 2024-04-25T11:01:17.388 INFO:teuthology.orchestra.run.smithi069.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49578.log.gz 2024-04-25T11:01:17.388 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43640.log 2024-04-25T11:01:17.388 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.40030.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40030.log.gz 2024-04-25T11:01:17.388 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48781.log 2024-04-25T11:01:17.388 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43682.log 2024-04-25T11:01:17.388 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.43640.log: /var/log/ceph/ceph-client.admin.48781.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43640.log.gz 2024-04-25T11:01:17.389 INFO:teuthology.orchestra.run.smithi069.stderr: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45239.log 2024-04-25T11:01:17.389 INFO:teuthology.orchestra.run.smithi069.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48781.log.gz 2024-04-25T11:01:17.389 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.43682.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43682.log.gz 2024-04-25T11:01:17.389 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49764.log 2024-04-25T11:01:17.389 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28622.log 2024-04-25T11:01:17.389 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.45239.log: /var/log/ceph/ceph-client.admin.49764.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45239.log.gz 2024-04-25T11:01:17.389 INFO:teuthology.orchestra.run.smithi069.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49764.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44312.log 2024-04-25T11:01:17.389 INFO:teuthology.orchestra.run.smithi069.stderr: 2024-04-25T11:01:17.389 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph.tmp-client.admin.28622.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28622.log.gz 2024-04-25T11:01:17.389 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46626.log 2024-04-25T11:01:17.389 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48477.log 2024-04-25T11:01:17.389 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.44312.log: /var/log/ceph/ceph-client.admin.46626.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44312.log.gz 2024-04-25T11:01:17.389 INFO:teuthology.orchestra.run.smithi069.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46626.log.gz 2024-04-25T11:01:17.390 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49178.log 2024-04-25T11:01:17.390 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.48477.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48477.log.gz 2024-04-25T11:01:17.390 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46933.log 2024-04-25T11:01:17.390 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50016.log 2024-04-25T11:01:17.390 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.49178.log: /var/log/ceph/ceph-client.admin.46933.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49178.log.gz 2024-04-25T11:01:17.390 INFO:teuthology.orchestra.run.smithi069.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46933.log.gz 2024-04-25T11:01:17.390 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.50016.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50016.log.gz 2024-04-25T11:01:17.390 INFO:teuthology.orchestra.run.smithi069.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T11:01:17.390 INFO:teuthology.orchestra.run.smithi069.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T11:01:17.391 INFO:teuthology.orchestra.run.smithi069.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T11:01:17.391 INFO:teuthology.orchestra.run.smithi146.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T11:01:17.391 INFO:teuthology.orchestra.run.smithi146.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T11:01:17.392 INFO:teuthology.orchestra.run.smithi069.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T11:01:17.394 INFO:teuthology.orchestra.run.smithi146.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T11:01:17.400 INFO:teuthology.orchestra.run.smithi069.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T11:01:17.411 INFO:teuthology.orchestra.run.smithi146.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T11:01:17.413 INFO:teuthology.orchestra.run.smithi146.stderr: 2024-04-25T11:01:17.413 INFO:teuthology.orchestra.run.smithi146.stderr:real 0m0.164s 2024-04-25T11:01:17.413 INFO:teuthology.orchestra.run.smithi146.stderr:user 0m0.697s 2024-04-25T11:01:17.413 INFO:teuthology.orchestra.run.smithi146.stderr:sys 0m0.048s 2024-04-25T11:01:17.449 INFO:teuthology.orchestra.run.smithi069.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T11:01:17.451 INFO:teuthology.orchestra.run.smithi069.stderr: 2024-04-25T11:01:17.451 INFO:teuthology.orchestra.run.smithi069.stderr:real 0m0.228s 2024-04-25T11:01:17.451 INFO:teuthology.orchestra.run.smithi069.stderr:user 0m0.812s 2024-04-25T11:01:17.451 INFO:teuthology.orchestra.run.smithi069.stderr:sys 0m0.156s 2024-04-25T11:01:17.451 INFO:tasks.ceph:Archiving logs... 2024-04-25T11:01:17.451 DEBUG:teuthology.misc:Transferring archived files from smithi069:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673109/remote/smithi069/log 2024-04-25T11:01:17.452 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T11:01:17.796 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673109/remote/smithi146/log 2024-04-25T11:01:17.797 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T11:01:17.923 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T11:01:17.947 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T11:01:17.947 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T11:01:17.949 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T11:01:18.021 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T11:01:18.030 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T11:01:18.030 DEBUG:teuthology.orchestra.run.smithi069:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:01:18.032 DEBUG:teuthology.orchestra.run.smithi146:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:01:18.044 INFO:teuthology.orchestra.run.smithi069.stderr:bash: line 1: ntpq: command not found 2024-04-25T11:01:18.046 INFO:teuthology.orchestra.run.smithi146.stderr:bash: line 1: ntpq: command not found 2024-04-25T11:01:18.048 INFO:teuthology.orchestra.run.smithi069.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T11:01:18.048 INFO:teuthology.orchestra.run.smithi069.stdout:=============================================================================== 2024-04-25T11:01:18.048 INFO:teuthology.orchestra.run.smithi069.stdout:^* hv01.front.sepia.ceph.com 3 6 377 10 +1075us[+1061us] +/- 56ms 2024-04-25T11:01:18.048 INFO:teuthology.orchestra.run.smithi069.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 10 -364us[ -378us] +/- 63ms 2024-04-25T11:01:18.048 INFO:teuthology.orchestra.run.smithi069.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 9 -946us[ -946us] +/- 70ms 2024-04-25T11:01:18.048 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:01:18.050 INFO:teuthology.orchestra.run.smithi146.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T11:01:18.050 INFO:teuthology.orchestra.run.smithi146.stdout:=============================================================================== 2024-04-25T11:01:18.050 INFO:teuthology.orchestra.run.smithi146.stdout:^* hv01.front.sepia.ceph.com 3 6 377 11 +1078us[+1071us] +/- 56ms 2024-04-25T11:01:18.050 INFO:teuthology.orchestra.run.smithi146.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 10 -358us[ -358us] +/- 63ms 2024-04-25T11:01:18.050 INFO:teuthology.orchestra.run.smithi146.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 -944us[ -950us] +/- 70ms 2024-04-25T11:01:18.051 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:01:18.051 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T11:01:18.064 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T11:01:18.065 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T11:01:18.109 DEBUG:teuthology.orchestra.run.smithi069:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T11:01:18.173 DEBUG:teuthology.orchestra.run.smithi146:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T11:01:18.234 DEBUG:teuthology.orchestra.run.smithi069:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T11:01:18.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:01:18.261 DEBUG:teuthology.orchestra.run.smithi146:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T11:01:18.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:01:18.287 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T11:01:18.296 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T11:01:18.334 INFO:teuthology.task.internal:Duration was 482.718417 seconds 2024-04-25T11:01:18.335 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T11:01:18.392 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T11:01:18.392 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T11:01:18.395 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T11:01:18.428 INFO:teuthology.orchestra.run.smithi069.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T11:01:18.432 INFO:teuthology.orchestra.run.smithi146.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T11:01:18.865 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T11:01:18.865 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi069.front.sepia.ceph.com 2024-04-25T11:01:18.865 DEBUG:teuthology.orchestra.run.smithi069:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T11:01:18.885 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi146.front.sepia.ceph.com 2024-04-25T11:01:18.885 DEBUG:teuthology.orchestra.run.smithi146:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T11:01:18.926 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T11:01:18.927 DEBUG:teuthology.orchestra.run.smithi069:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T11:01:18.929 DEBUG:teuthology.orchestra.run.smithi146:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T11:01:18.992 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T11:01:18.993 DEBUG:teuthology.orchestra.run.smithi069:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T11:01:18.995 DEBUG:teuthology.orchestra.run.smithi146:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T11:01:19.131 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T11:01:19.158 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T11:01:19.158 DEBUG:teuthology.orchestra.run.smithi069:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T11:01:19.182 DEBUG:teuthology.orchestra.run.smithi146:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T11:01:19.205 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T11:01:19.215 DEBUG:teuthology.orchestra.run.smithi069:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:01:19.225 DEBUG:teuthology.orchestra.run.smithi146:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:01:19.244 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern = core 2024-04-25T11:01:19.267 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern = core 2024-04-25T11:01:19.279 DEBUG:teuthology.orchestra.run.smithi069:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:01:19.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:01:19.311 DEBUG:teuthology.orchestra.run.smithi146:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:01:19.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:01:19.334 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T11:01:19.347 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T11:01:19.348 DEBUG:teuthology.misc:Transferring archived files from smithi069:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673109/remote/smithi069 2024-04-25T11:01:19.348 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T11:01:19.386 DEBUG:teuthology.misc:Transferring archived files from smithi146:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673109/remote/smithi146 2024-04-25T11:01:19.386 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T11:01:19.419 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T11:01:19.419 DEBUG:teuthology.orchestra.run.smithi069:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T11:01:19.423 DEBUG:teuthology.orchestra.run.smithi146:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T11:01:19.468 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T11:01:19.479 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T11:01:19.480 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T11:01:19.593 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T11:01:19.593 DEBUG:teuthology.orchestra.run.smithi069:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T11:01:19.596 DEBUG:teuthology.orchestra.run.smithi146:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T11:01:19.608 INFO:teuthology.orchestra.run.smithi069.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 11:01 /home/ubuntu/cephtest 2024-04-25T11:01:19.610 INFO:teuthology.orchestra.run.smithi146.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 11:01 /home/ubuntu/cephtest 2024-04-25T11:01:19.611 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T11:01:19.644 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T11:01:19.745 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/cap-flush} duration: 482.7184171676636 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-04-25T11:01:19.745 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T11:01:19.828 INFO:teuthology.run:pass